Just One G - II
Setup
#Packages
library(pacman)
p_load(dplyr, DT, meta, DescTools, ggplot2, metafor, dmetar)
#Ad hoc functions
r2z <- function(r){
z = 0.5*log((1+r)/(1-r))
return(z)}
z2r <- function(z){
r = (exp(1)^(2*z)-1)/(exp(1)^(2*z)+1)
return(r)}
#Data
datatable(JOG, extensions = c("Buttons", "FixedColumns"), options = list(dom = 'Bfrtip', buttons = c('copy', 'csv', 'print'), scrollX = T, fixedColumns = list(leftColumns = 3)))Background
This is a meta-analysis of joint confirmatory factor analytic assessments of the indifference of the indicator. It will be expanded later to include a handful of new data points and results from other methods. These analyses have already been conducted and their results are in line with that presented below.
Analysis
JCFAs
Located at https://rpubs.com/JLLJ/JOG, https://rpubs.com/JLLJ/NJDH, https://rpubs.com/JLLJ/PIAPSY and https://rpubs.com/JLLJ/FRE20. The data from the reanalysis of Wong et al. (1995) was omitted due to NPD.
Meta-analytic Results
Correlations of one or greater (i.e., ultra-Heywood cases or extreme multicollinearity) were coded as 0.99 but coding as 0.999 is arguably better, though it would have made the results more extremely favorable to the correlation being higher because of Fisher’s r-to-Z transformation. Buczylowska, Petermann & Daseking’s (2020) study produced a coefficient of exactly 0.999, so it was coded as such, even though it should be considered further from 1 than the coefficients exceeding it or capable of being matched to it (which it was). Results with coding from 0.99 to 0.999 in increments of 0.001 were assessed and generated linear improvements.
#r-to-z transformed for proper variance estimation
ZCOR <- metacor(rG,
sqrt(n),
data = JOG,
studlab = SY,
sm = "ZCOR",
method.tau = "SJ")
ZCOR## COR 95%-CI %W(fixed)
## DeVries & Kohlberg1974 0.9030 [ 0.4543; 0.9862] 0.2
## Devries 1974 A 0.8120 [ 0.1601; 0.9707] 0.3
## Devries1974 B 0.7880 [ 0.2528; 0.9539] 0.4
## Hathaway1972 A 0.8920 [ 0.4668; 0.9822] 0.3
## Hathaway1972 B 0.9240 [ 0.5980; 0.9877] 0.3
## Rindermann & Ackermann2020 A 0.5750 [-0.4079; 0.9406] 0.2
## Rindermann & Ackermann2020 B 0.7640 [ 0.1840; 0.9494] 0.4
## Lim1988 0.9040 [ 0.7767; 0.9603] 1.2
## Stone1992 0.9900 [ 0.9596; 0.9976] 0.5
## Byrd & Buckhalt1991 0.9900 [ 0.9273; 0.9987] 0.2
## Tirre & Field2002 A 0.9900 [ 0.9728; 0.9963] 1.0
## Tirre & Field2002 B 0.9900 [ 0.9732; 0.9963] 1.0
## Tirre & Field2002 C 0.9730 [ 0.9109; 0.9920] 0.7
## Wothke et al.1991 0.9900 [ 0.9695; 0.9967] 0.8
## Williamson1969 0.9900 [ 0.9680; 0.9969] 0.7
## Kettner1976 A 0.9510 [ 0.8665; 0.9825] 0.9
## Kettner1976 B 0.9900 [ 0.9718; 0.9965] 0.9
## Palmer et al.1990 0.9750 [ 0.9360; 0.9904] 1.1
## Deary et al.1989 0.6100 [ 0.0120; 0.8866] 0.5
## Kranzler & Jensen1991 0.3680 [-0.3382; 0.8091] 0.5
## Luo, Thompson & Detterman2003 0.8700 [ 0.7141; 0.9437] 1.3
## Carey1992 0.8900 [ 0.7889; 0.9442] 2.0
## Abrahams et al.1994 0.8610 [ 0.7979; 0.9054] 6.0
## Wolfe et al.1995 0.8660 [ 0.8084; 0.9072] 6.6
## Naglieri & Jensen1987 0.9900 [ 0.9661; 0.9971] 0.7
## Engelhardt2018 0.9800 [ 0.9569; 0.9908] 1.6
## Stauffer, Ree & Carretta1996 0.9940 [ 0.9832; 0.9979] 0.9
## Keith, Kranzler & Flanagan2001 0.9900 [ 0.9647; 0.9972] 0.6
## Deary et al.2004 0.9200 [ 0.6052; 0.9860] 0.3
## Acton & Schroeder2001 0.6780 [ 0.4203; 0.8345] 1.7
## Meyer et al.2010 0.7800 [ 0.6085; 0.8819] 2.2
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 1.2
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 1.2
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 1.2
## Deary et al.2007 0.8100 [ 0.7361; 0.8648] 7.3
## Quiroga et al.2019 0.8320 [ 0.4818; 0.9530] 0.6
## Floyd et al.2013 0.9700 [ 0.9061; 0.9906] 0.7
## Floyd et al.2013 0.9500 [ 0.8297; 0.9860] 0.6
## Floyd et al.2013 0.9900 [ 0.9625; 0.9974] 0.6
## Floyd et al.2013 0.8900 [ 0.6162; 0.9719] 0.5
## Floyd et al.2013 0.9200 [ 0.6619; 0.9831] 0.4
## Peters, Kyngdon & Stillwell2021 0.7200 [ 0.2257; 0.9195] 0.5
## Reynolds et al.2015 0.8400 [ 0.5355; 0.9512] 0.6
## Valerius & Sparfeldt2014 A 0.9900 [ 0.9765; 0.9958] 1.3
## Valerius & Sparfeldt2014 B 0.9200 [ 0.8205; 0.9654] 1.3
## Valerius & Sparfeldt2014 C 0.9500 [ 0.8856; 0.9786] 1.3
## Quiroga et al.2015 0.9300 [ 0.7855; 0.9783] 0.7
## Swagerman et al.2016 0.9900 [ 0.9702; 0.9967] 0.8
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.3
## Johnson, te Nijenhuis & Bouchard2008 0.8900 [ 0.7515; 0.9534] 1.3
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.3
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.3
## Johnson, te Nijenhuis & Bouchard2008 0.8100 [ 0.5926; 0.9174] 1.3
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.3
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.3
## Johnson, te Nijenhuis & Bouchard2008 0.7300 [ 0.4489; 0.8796] 1.3
## Johnson, te Nijenhuis & Bouchard2008 0.9300 [ 0.8376; 0.9707] 1.3
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.3
## Floyd et al.2010 0.9900 [ 0.9567; 0.9977] 0.5
## Kaufman et al.2012 0.8600 [ 0.7650; 0.9184] 3.1
## Kaufman et al.2012 0.8000 [ 0.6963; 0.8710] 4.4
## Salthouse2013 0.9100 [ 0.6399; 0.9800] 0.4
## Flanagan2000 0.8300 [ 0.5115; 0.9480] 0.6
## Floyd et al.2007 0.7770 [ 0.6472; 0.8630] 3.5
## Parkin & Beaujean2012 0.9100 [ 0.7984; 0.9612] 1.3
## Vanderwood et al.2002 0.9440 [ 0.9070; 0.9665] 3.6
## Freis et al.2020 0.7040 [ 0.5912; 0.7898] 6.5
## Buczylowska, Petermann & Daseking2020 0.9990 [ 0.9968; 0.9997] 0.7
## Wang et al.2021 0.5960 [ 0.1125; 0.8513] 0.8
## Snow et al.1977 0.9900 [ 0.9732; 0.9963] 1.0
## McGrew & Woodcock2001 A 0.9900 [ 0.9641; 0.9972] 0.6
## McGrew & Woodcock2001 B 0.9900 [ 0.9641; 0.9972] 0.6
## McGrew & Woodcock2001 C 0.9590 [ 0.8584; 0.9886] 0.6
## McGrew & Woodcock2001 D 0.9900 [ 0.9640; 0.9973] 0.6
## Woodcock1978 0.9790 [ 0.9290; 0.9939] 0.6
## Undheim1976 0.9900 [ 0.9636; 0.9973] 0.6
## Grigorenko et al.2004 0.6810 [ 0.2827; 0.8790] 0.9
## %W(random)
## DeVries & Kohlberg1974 1.0
## Devries 1974 A 1.0
## Devries1974 B 1.1
## Hathaway1972 A 1.0
## Hathaway1972 B 1.0
## Rindermann & Ackermann2020 A 0.9
## Rindermann & Ackermann2020 B 1.1
## Lim1988 1.4
## Stone1992 1.2
## Byrd & Buckhalt1991 1.0
## Tirre & Field2002 A 1.3
## Tirre & Field2002 B 1.3
## Tirre & Field2002 C 1.3
## Wothke et al.1991 1.3
## Williamson1969 1.3
## Kettner1976 A 1.3
## Kettner1976 B 1.3
## Palmer et al.1990 1.4
## Deary et al.1989 1.2
## Kranzler & Jensen1991 1.2
## Luo, Thompson & Detterman2003 1.4
## Carey1992 1.4
## Abrahams et al.1994 1.5
## Wolfe et al.1995 1.5
## Naglieri & Jensen1987 1.3
## Engelhardt2018 1.4
## Stauffer, Ree & Carretta1996 1.3
## Keith, Kranzler & Flanagan2001 1.3
## Deary et al.2004 1.1
## Acton & Schroeder2001 1.4
## Meyer et al.2010 1.4
## Johnson et al.2004 1.4
## Johnson et al.2004 1.4
## Johnson et al.2004 1.4
## Deary et al.2007 1.5
## Quiroga et al.2019 1.2
## Floyd et al.2013 1.3
## Floyd et al.2013 1.2
## Floyd et al.2013 1.2
## Floyd et al.2013 1.2
## Floyd et al.2013 1.1
## Peters, Kyngdon & Stillwell2021 1.2
## Reynolds et al.2015 1.3
## Valerius & Sparfeldt2014 A 1.4
## Valerius & Sparfeldt2014 B 1.4
## Valerius & Sparfeldt2014 C 1.4
## Quiroga et al.2015 1.3
## Swagerman et al.2016 1.3
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Floyd et al.2010 1.2
## Kaufman et al.2012 1.5
## Kaufman et al.2012 1.5
## Salthouse2013 1.2
## Flanagan2000 1.3
## Floyd et al.2007 1.5
## Parkin & Beaujean2012 1.4
## Vanderwood et al.2002 1.5
## Freis et al.2020 1.5
## Buczylowska, Petermann & Daseking2020 1.3
## Wang et al.2021 1.3
## Snow et al.1977 1.3
## McGrew & Woodcock2001 A 1.2
## McGrew & Woodcock2001 B 1.2
## McGrew & Woodcock2001 C 1.2
## McGrew & Woodcock2001 D 1.2
## Woodcock1978 1.3
## Undheim1976 1.2
## Grigorenko et al.2004 1.3
##
## Number of studies combined: k = 77
##
## COR 95%-CI z p-value
## Fixed effect model 0.9302 [0.9231; 0.9366] 65.23 0
## Random effects model 0.9541 [0.9363; 0.9670] 21.89 < 0.0001
##
## Quantifying heterogeneity:
## tau^2 = 0.4796 [0.3286; 0.6975]; tau = 0.6926 [0.5732; 0.8352];
## I^2 = 89.7% [87.8%; 91.3%]; H = 3.12 [2.86; 3.39]
##
## Test of heterogeneity:
## Q d.f. p-value
## 738.68 76 < 0.0001
##
## Details on meta-analytical method:
## - Inverse variance method
## - Sidik-Jonkman estimator for tau^2
## - Q-profile method for confidence interval of tau^2 and tau
## - Fisher's z transformation of correlationsRMADAT <- escalc(measure = "ZCOR", ri = rG, ni = n, data = JOG)
ZCORRMA <- rma.uni(yi = yi, vi = vi, data = RMADAT, method = "ML"); ZCORRMA##
## Random-Effects Model (k = 77; tau^2 estimator: ML)
##
## tau^2 (estimated amount of total heterogeneity): 0.5441 (SE = 0.0886)
## tau (square root of estimated tau^2 value): 0.7376
## I^2 (total heterogeneity / total variability): 99.81%
## H^2 (total variability / sampling variability): 516.83
##
## Test for Heterogeneity:
## Q(df = 76) = 22413.7875, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 1.8725 0.0845 22.1587 <.0001 1.7069 2.0381 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1z2r(ZCORRMA$b)## [,1]
## intrcpt 0.9538215forest(ZCOR,
sortvar = TE,
xlim = c(-1, 1),
rightlabs = c("Correlation", "95% CI", "Weight"),
leftcols = c("Source"),
leftlabs = c("Study"),
pooled.totals = F,
smlab = "",
text.random = "Overall Effect",
print.tau2 = F,
col.diamond = "gold",
col.diamond.lines = "black",
col.predict = "black",
print.I2.ci = F,
digits.sd = 2,
comb.fixed = F,
col.square = "#5F85E7",
overall = T)
Publication Bias
Methods to be used here: Failsafe-N (not a publication bias method, but typically used as one), funnel plots, Egger’s test, Trim-and-Fill, p-curve, and reestimation with the the Henmi & Copas approach. The 10% highest precision effects are not meaningful here because of range restriction.
fsn(yi = yi, vi = vi, data = RMADAT, type = "Rosenthal"); fsn(yi = yi, vi = vi, data = RMADAT, type = "Rosenberg")##
## Fail-safe N Calculation Using the Rosenthal Approach
##
## Observed Significance Level: <.0001
## Target Significance Level: 0.05
##
## Fail-safe N: 3286147##
## Fail-safe N Calculation Using the Rosenberg Approach
##
## Average Effect Size: 1.3825
## Observed Significance Level: <.0001
## Target Significance Level: 0.05
##
## Fail-safe N: 3029351OrwinRange <- seq(0.01, 1.22, 0.01) #1.22 is chosen as the endpoint since it corresponds to r = 0.84, or r = 0.01 less than the typical cutoff for convergent validity/lack of discriminant validity of r = 0.85 (though, in doing this, I am not recommending strict cutoffs be used).
fsn(yi = yi, vi = vi, data = RMADAT, type = "Orwin", target = OrwinRange); fsn(yi = yi, vi = vi, data = RMADAT, type = "Orwin", target = 1.22)##
## Fail-safe N Calculation Using the Orwin Approach
##
## Average Effect Size: 1.8713
## Target Effect Size: 0.0100 0.0200 0.0300 0.0400 0.0500 0.0600 0.0700 0.0800 0.0900 0.1000 0.1100 0.1200 0.1300 0.1400 0.1500 0.1600 0.1700 0.1800 0.1900 0.2000 0.2100 0.2200 0.2300 0.2400 0.2500 0.2600 0.2700 0.2800 0.2900 0.3000 0.3100 0.3200 0.3300 0.3400 0.3500 0.3600 0.3700 0.3800 0.3900 0.4000 0.4100 0.4200 0.4300 0.4400 0.4500 0.4600 0.4700 0.4800 0.4900 0.5000 0.5100 0.5200 0.5300 0.5400 0.5500 0.5600 0.5700 0.5800 0.5900 0.6000 0.6100 0.6200 0.6300 0.6400 0.6500 0.6600 0.6700 0.6800 0.6900 0.7000 0.7100 0.7200 0.7300 0.7400 0.7500 0.7600 0.7700 0.7800 0.7900 0.8000 0.8100 0.8200 0.8300 0.8400 0.8500 0.8600 0.8700 0.8800 0.8900 0.9000 0.9100 0.9200 0.9300 0.9400 0.9500 0.9600 0.9700 0.9800 0.9900 1.0000 1.0100 1.0200 1.0300 1.0400 1.0500 1.0600 1.0700 1.0800 1.0900 1.1000 1.1100 1.1200 1.1300 1.1400 1.1500 1.1600 1.1700 1.1800 1.1900 1.2000 1.2100 1.2200
##
## Fail-safe N: 14333##
## Fail-safe N Calculation Using the Orwin Approach
##
## Average Effect Size: 1.8713
## Target Effect Size: 1.2200
##
## Fail-safe N: 42dfZ <- data.frame(ZCOR$seTE, ZCOR$TE); dfZ$se <- FisherZInv(dfZ$ZCOR.seTE); dfZ$r <- FisherZInv(dfZ$ZCOR.TE)
estimateT = FisherZInv(ZCOR$TE.random); set = FisherZInv(ZCOR$seTE.random); estimateT## [1] 0.9540834dfZse.seq = seq(0, max(dfZ$se), 0.001)
ll95 = estimateT - (1.96*se.seq)
ul95 = estimateT + (1.96*se.seq)
ll95a = FisherZInv(ZCOR$lower.random)
ul95a = FisherZInv(ZCOR$upper.random)
ll99 = estimateT - (3.29*se.seq)
ul99 = estimateT + (3.29*se.seq)
ll99a = 1.67857*FisherZInv(ZCOR$lower.random)
ul99a = 1.67857*FisherZInv(ZCOR$upper.random)
meanll95 = estimateT - (1.96*set)
meanul95 = estimateT + (1.96*set)
dfZCI <- data.frame(ll95, ul95, ll99, ul99, se.seq, estimateT, meanll95, meanul95, ll95a, ul95a, ll99a, ul99a)
STAND <- ggplot(aes(x = se, y = r), data = dfZ) +
geom_point(shape = 16, size = 3, colour = "#00348E") +
xlab('Standard Error') + ylab('z-to-r Transformed Correlations') +
geom_line(aes(x = se.seq, y = ll95), linetype = 'dotted', colour = "#666666", size = 1, data = dfZCI) +
geom_line(aes(x = se.seq, y = ul95), linetype = 'dotted', colour = "#666666", size = 1, data = dfZCI) +
geom_line(aes(x = se.seq, y = ll99), linetype = 'dashed', colour = "#666666", size = 1, data = dfZCI) +
geom_line(aes(x = se.seq, y = ul99), linetype = 'dashed', colour = "#666666", size = 1, data = dfZCI) +
geom_segment(aes(x = min(se.seq), y = estimateT, xend = max(se.seq), yend = estimateT), linetype='dotted', colour = "#E9C535", size = 1, data=dfZCI) +
geom_segment(aes(x = min(se.seq), y = ll95a, xend = max(se.seq), yend = ll95a), linetype='dotted' , colour = "gold", size = 1, data=dfZCI) +
geom_segment(aes(x = min(se.seq), y = ul95a, xend = max(se.seq), yend = ul95a), linetype='dotted' , colour = "gold", size = 1, data=dfZCI) +
scale_x_reverse() +
coord_flip() +
theme_bw() +
theme(text = element_text(family = "serif", size = 12))
STAND
regtest(ZCORRMA)##
## Regression Test for Funnel Plot Asymmetry
##
## model: mixed-effects meta-regression model
## predictor: standard error
##
## test for funnel plot asymmetry: z = -0.3420, p = 0.7324TF <- trimfill(ZCORRMA, side = "left"); TF; funnel(TF, legend = T); z2r(TF$b)##
## Estimated number of missing studies on the left side: 5 (SE = 5.5257)
##
## Random-Effects Model (k = 82; tau^2 estimator: ML)
##
## tau^2 (estimated amount of total heterogeneity): 0.6058 (SE = 0.0955)
## tau (square root of estimated tau^2 value): 0.7784
## I^2 (total heterogeneity / total variability): 99.82%
## H^2 (total variability / sampling variability): 547.49
##
## Test for Heterogeneity:
## Q(df = 81) = 23133.8030, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 1.7990 0.0864 20.8303 <.0001 1.6298 1.9683 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## [,1]
## intrcpt 0.9467071TF <- trimfill(ZCORRMA, side = "right"); TF; funnel(TF, legend = T); z2r(TF$b)##
## Estimated number of missing studies on the right side: 0 (SE = 5.0352)
##
## Random-Effects Model (k = 77; tau^2 estimator: ML)
##
## tau^2 (estimated amount of total heterogeneity): 0.5441 (SE = 0.0886)
## tau (square root of estimated tau^2 value): 0.7376
## I^2 (total heterogeneity / total variability): 99.81%
## H^2 (total variability / sampling variability): 516.83
##
## Test for Heterogeneity:
## Q(df = 76) = 22413.7875, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 1.8725 0.0845 22.1587 <.0001 1.7069 2.0381 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## [,1]
## intrcpt 0.9538215TF <- trimfill(ZCORRMA, side = "right"); TF; z2r(TF$b) #This is the one where the side is automatically chosen based on Egger's test.##
## Estimated number of missing studies on the right side: 0 (SE = 5.0352)
##
## Random-Effects Model (k = 77; tau^2 estimator: ML)
##
## tau^2 (estimated amount of total heterogeneity): 0.5441 (SE = 0.0886)
## tau (square root of estimated tau^2 value): 0.7376
## I^2 (total heterogeneity / total variability): 99.81%
## H^2 (total variability / sampling variability): 516.83
##
## Test for Heterogeneity:
## Q(df = 76) = 22413.7875, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 1.8725 0.0845 22.1587 <.0001 1.7069 2.0381 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1## [,1]
## intrcpt 0.9538215pcurve(ZCOR, effect.estimation = T, N = ZCOR$n, dmax = 4)## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## Warning in ks.test(ppr, punif): ties should not be present for the Kolmogorov-
## Smirnov test
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 77
## - Total number of p<0.05 studies included into the analysis: k = 75 (97.4%)
## - Total number of studies with p<0.025: k = 74 (96.1%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0 -49.726 0 -48.674 0
## Flatness test 1 44.720 1 46.783 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 99% (99%-99%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
## P-curve's estimate of the true effect size: d=2.982z2r(2.982)## [1] 0.9948739HCZ <- hc(ZCORRMA); HCZ; z2r(HCZ$beta)##
## method tau2 estimate se ci.lb ci.ub
## rma ML 0.5441 1.8725 0.0845 1.7069 2.0381
## hc DL 0.3100 1.3825 0.1659 1.0434 1.7216## [1] 0.8815144Heterogeneity
Methods to be used here: Outlier detection and removal, with outlier classification based on residence outside of the study’s 95% CI (r = 0.94 to 0.97), influence analyses, radial plotting, and Graphic Display of Heterogeneity. There is no need to assess whether there was attenuation by unreliability and its effect because the correlations come from SEMs and the correlations used are between latent variables.
ZCOR$lower.random; ZCOR$upper.random## [1] 1.707527## [1] 2.043323ZCORRMA$ci.lb; ZCORRMA$ci.ub## [1] 1.706888## [1] 2.038141find.outliers(ZCOR)## Identified outliers (fixed-effect model)
## ----------------------------------------
## "Stone1992", "Tirre & Field2002 A", "Tirre & Field2002 B", "Wothke et al.1991", "Williamson1969", "Kettner1976 B", "Deary et al.1989", "Kranzler & Jensen1991", "Abrahams et al.1994", "Wolfe et al.1995", "Naglieri & Jensen1987", "Engelhardt2018", "Stauffer, Ree & Carretta1996", "Keith, Kranzler & Flanagan2001", "Acton & Schroeder2001", "Meyer et al.2010", "Johnson et al.2004", "Johnson et al.2004", "Johnson et al.2004", "Deary et al.2007", "Floyd et al.2013", "Peters, Kyngdon & Stillwell2021", "Valerius & Sparfeldt2014 A", "Swagerman et al.2016", "Johnson, te Nijenhuis & Bouchard2008", "Johnson, te Nijenhuis & Bouchard2008", "Johnson, te Nijenhuis & Bouchard2008", "Johnson, te Nijenhuis & Bouchard2008", "Johnson, te Nijenhuis & Bouchard2008", "Johnson, te Nijenhuis & Bouchard2008", "Johnson, te Nijenhuis & Bouchard2008", "Johnson, te Nijenhuis & Bouchard2008", "Floyd et al.2010", "Kaufman et al.2012", "Kaufman et al.2012", "Floyd et al.2007", "Freis et al.2020", "Buczylowska, Petermann & Daseking2020", "Wang et al.2021", "Snow et al.1977", "McGrew & Woodcock2001 A", "McGrew & Woodcock2001 B", "McGrew & Woodcock2001 D", "Undheim1976", "Grigorenko et al.2004"
##
## Results with outliers removed
## -----------------------------
## COR 95%-CI %W(fixed)
## DeVries & Kohlberg1974 0.9030 [ 0.4543; 0.9862] 1.0
## Devries 1974 A 0.8120 [ 0.1601; 0.9707] 1.0
## Devries1974 B 0.7880 [ 0.2528; 0.9539] 1.5
## Hathaway1972 A 0.8920 [ 0.4668; 0.9822] 1.1
## Hathaway1972 B 0.9240 [ 0.5980; 0.9877] 1.1
## Rindermann & Ackermann2020 A 0.5750 [-0.4079; 0.9406] 0.8
## Rindermann & Ackermann2020 B 0.7640 [ 0.1840; 0.9494] 1.4
## Lim1988 0.9040 [ 0.7767; 0.9603] 4.5
## Stone1992 0.9900 [ 0.9596; 0.9976] 0.0
## Byrd & Buckhalt1991 0.9900 [ 0.9273; 0.9987] 0.9
## Tirre & Field2002 A 0.9900 [ 0.9728; 0.9963] 0.0
## Tirre & Field2002 B 0.9900 [ 0.9732; 0.9963] 0.0
## Tirre & Field2002 C 0.9730 [ 0.9109; 0.9920] 2.5
## Wothke et al.1991 0.9900 [ 0.9695; 0.9967] 0.0
## Williamson1969 0.9900 [ 0.9680; 0.9969] 0.0
## Kettner1976 A 0.9510 [ 0.8665; 0.9825] 3.5
## Kettner1976 B 0.9900 [ 0.9718; 0.9965] 0.0
## Palmer et al.1990 0.9750 [ 0.9360; 0.9904] 4.1
## Deary et al.1989 0.6100 [ 0.0120; 0.8866] 0.0
## Kranzler & Jensen1991 0.3680 [-0.3382; 0.8091] 0.0
## Luo, Thompson & Detterman2003 0.8700 [ 0.7141; 0.9437] 4.9
## Carey1992 0.8900 [ 0.7889; 0.9442] 7.6
## Abrahams et al.1994 0.8610 [ 0.7979; 0.9054] 0.0
## Wolfe et al.1995 0.8660 [ 0.8084; 0.9072] 0.0
## Naglieri & Jensen1987 0.9900 [ 0.9661; 0.9971] 0.0
## Engelhardt2018 0.9800 [ 0.9569; 0.9908] 0.0
## Stauffer, Ree & Carretta1996 0.9940 [ 0.9832; 0.9979] 0.0
## Keith, Kranzler & Flanagan2001 0.9900 [ 0.9647; 0.9972] 0.0
## Deary et al.2004 0.9200 [ 0.6052; 0.9860] 1.2
## Acton & Schroeder2001 0.6780 [ 0.4203; 0.8345] 0.0
## Meyer et al.2010 0.7800 [ 0.6085; 0.8819] 0.0
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 0.0
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 0.0
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 0.0
## Deary et al.2007 0.8100 [ 0.7361; 0.8648] 0.0
## Quiroga et al.2019 0.8320 [ 0.4818; 0.9530] 2.1
## Floyd et al.2013 0.9700 [ 0.9061; 0.9906] 2.7
## Floyd et al.2013 0.9500 [ 0.8297; 0.9860] 2.3
## Floyd et al.2013 0.9900 [ 0.9625; 0.9974] 0.0
## Floyd et al.2013 0.8900 [ 0.6162; 0.9719] 1.9
## Floyd et al.2013 0.9200 [ 0.6619; 0.9831] 1.5
## Peters, Kyngdon & Stillwell2021 0.7200 [ 0.2257; 0.9195] 0.0
## Reynolds et al.2015 0.8400 [ 0.5355; 0.9512] 2.4
## Valerius & Sparfeldt2014 A 0.9900 [ 0.9765; 0.9958] 0.0
## Valerius & Sparfeldt2014 B 0.9200 [ 0.8205; 0.9654] 5.1
## Valerius & Sparfeldt2014 C 0.9500 [ 0.8856; 0.9786] 5.1
## Quiroga et al.2015 0.9300 [ 0.7855; 0.9783] 2.6
## Swagerman et al.2016 0.9900 [ 0.9702; 0.9967] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.8900 [ 0.7515; 0.9534] 4.8
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.8100 [ 0.5926; 0.9174] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.7300 [ 0.4489; 0.8796] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.9300 [ 0.8376; 0.9707] 4.8
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 0.0
## Floyd et al.2010 0.9900 [ 0.9567; 0.9977] 0.0
## Kaufman et al.2012 0.8600 [ 0.7650; 0.9184] 0.0
## Kaufman et al.2012 0.8000 [ 0.6963; 0.8710] 0.0
## Salthouse2013 0.9100 [ 0.6399; 0.9800] 1.6
## Flanagan2000 0.8300 [ 0.5115; 0.9480] 2.4
## Floyd et al.2007 0.7770 [ 0.6472; 0.8630] 0.0
## Parkin & Beaujean2012 0.9100 [ 0.7984; 0.9612] 5.0
## Vanderwood et al.2002 0.9440 [ 0.9070; 0.9665] 13.7
## Freis et al.2020 0.7040 [ 0.5912; 0.7898] 0.0
## Buczylowska, Petermann & Daseking2020 0.9990 [ 0.9968; 0.9997] 0.0
## Wang et al.2021 0.5960 [ 0.1125; 0.8513] 0.0
## Snow et al.1977 0.9900 [ 0.9732; 0.9963] 0.0
## McGrew & Woodcock2001 A 0.9900 [ 0.9641; 0.9972] 0.0
## McGrew & Woodcock2001 B 0.9900 [ 0.9641; 0.9972] 0.0
## McGrew & Woodcock2001 C 0.9590 [ 0.8584; 0.9886] 2.3
## McGrew & Woodcock2001 D 0.9900 [ 0.9640; 0.9973] 0.0
## Woodcock1978 0.9790 [ 0.9290; 0.9939] 2.4
## Undheim1976 0.9900 [ 0.9636; 0.9973] 0.0
## Grigorenko et al.2004 0.6810 [ 0.2827; 0.8790] 0.0
## %W(random) exclude
## DeVries & Kohlberg1974 1.7
## Devries 1974 A 1.7
## Devries1974 B 2.3
## Hathaway1972 A 1.9
## Hathaway1972 B 1.9
## Rindermann & Ackermann2020 A 1.5
## Rindermann & Ackermann2020 B 2.2
## Lim1988 4.1
## Stone1992 0.0 *
## Byrd & Buckhalt1991 1.7
## Tirre & Field2002 A 0.0 *
## Tirre & Field2002 B 0.0 *
## Tirre & Field2002 C 3.2
## Wothke et al.1991 0.0 *
## Williamson1969 0.0 *
## Kettner1976 A 3.7
## Kettner1976 B 0.0 *
## Palmer et al.1990 4.0
## Deary et al.1989 0.0 *
## Kranzler & Jensen1991 0.0 *
## Luo, Thompson & Detterman2003 4.3
## Carey1992 4.9
## Abrahams et al.1994 0.0 *
## Wolfe et al.1995 0.0 *
## Naglieri & Jensen1987 0.0 *
## Engelhardt2018 0.0 *
## Stauffer, Ree & Carretta1996 0.0 *
## Keith, Kranzler & Flanagan2001 0.0 *
## Deary et al.2004 2.0
## Acton & Schroeder2001 0.0 *
## Meyer et al.2010 0.0 *
## Johnson et al.2004 0.0 *
## Johnson et al.2004 0.0 *
## Johnson et al.2004 0.0 *
## Deary et al.2007 0.0 *
## Quiroga et al.2019 2.9
## Floyd et al.2013 3.3
## Floyd et al.2013 3.0
## Floyd et al.2013 0.0 *
## Floyd et al.2013 2.7
## Floyd et al.2013 2.3
## Peters, Kyngdon & Stillwell2021 0.0 *
## Reynolds et al.2015 3.1
## Valerius & Sparfeldt2014 A 0.0 *
## Valerius & Sparfeldt2014 B 4.3
## Valerius & Sparfeldt2014 C 4.3
## Quiroga et al.2015 3.2
## Swagerman et al.2016 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 4.2
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 4.2
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Floyd et al.2010 0.0 *
## Kaufman et al.2012 0.0 *
## Kaufman et al.2012 0.0 *
## Salthouse2013 2.4
## Flanagan2000 3.1
## Floyd et al.2007 0.0 *
## Parkin & Beaujean2012 4.3
## Vanderwood et al.2002 5.6
## Freis et al.2020 0.0 *
## Buczylowska, Petermann & Daseking2020 0.0 *
## Wang et al.2021 0.0 *
## Snow et al.1977 0.0 *
## McGrew & Woodcock2001 A 0.0 *
## McGrew & Woodcock2001 B 0.0 *
## McGrew & Woodcock2001 C 3.0
## McGrew & Woodcock2001 D 0.0 *
## Woodcock1978 3.1
## Undheim1976 0.0 *
## Grigorenko et al.2004 0.0 *
##
## Number of studies combined: k = 32
##
## COR 95%-CI z p-value
## Fixed effect model 0.9261 [0.9110; 0.9388] 32.84 < 0.0001
## Random effects model 0.9237 [0.8986; 0.9428] 21.23 < 0.0001
##
## Quantifying heterogeneity:
## tau^2 = 0.0854 [0.0000; 0.1416]; tau = 0.2922 [0.0000; 0.3763];
## I^2 = 23.9% [0.0%; 51.0%]; H = 1.15 [1.00; 1.43]
##
## Test of heterogeneity:
## Q d.f. p-value
## 40.72 31 0.1136
##
## Details on meta-analytical method:
## - Inverse variance method
## - Sidik-Jonkman estimator for tau^2
## - Q-profile method for confidence interval of tau^2 and tau
## - Fisher's z transformation of correlations
##
## Identified outliers (random-effects model)
## ------------------------------------------
## "Tirre & Field2002 A", "Tirre & Field2002 B", "Wothke et al.1991", "Williamson1969", "Kettner1976 B", "Deary et al.1989", "Kranzler & Jensen1991", "Abrahams et al.1994", "Wolfe et al.1995", "Stauffer, Ree & Carretta1996", "Acton & Schroeder2001", "Meyer et al.2010", "Johnson et al.2004", "Johnson et al.2004", "Johnson et al.2004", "Deary et al.2007", "Peters, Kyngdon & Stillwell2021", "Valerius & Sparfeldt2014 A", "Swagerman et al.2016", "Johnson, te Nijenhuis & Bouchard2008", "Johnson, te Nijenhuis & Bouchard2008", "Johnson, te Nijenhuis & Bouchard2008", "Johnson, te Nijenhuis & Bouchard2008", "Johnson, te Nijenhuis & Bouchard2008", "Johnson, te Nijenhuis & Bouchard2008", "Johnson, te Nijenhuis & Bouchard2008", "Johnson, te Nijenhuis & Bouchard2008", "Kaufman et al.2012", "Kaufman et al.2012", "Floyd et al.2007", "Freis et al.2020", "Buczylowska, Petermann & Daseking2020", "Wang et al.2021", "Snow et al.1977", "Grigorenko et al.2004"
##
## Results with outliers removed
## -----------------------------
## COR 95%-CI %W(fixed)
## DeVries & Kohlberg1974 0.9030 [ 0.4543; 0.9862] 0.8
## Devries 1974 A 0.8120 [ 0.1601; 0.9707] 0.8
## Devries1974 B 0.7880 [ 0.2528; 0.9539] 1.2
## Hathaway1972 A 0.8920 [ 0.4668; 0.9822] 0.9
## Hathaway1972 B 0.9240 [ 0.5980; 0.9877] 0.9
## Rindermann & Ackermann2020 A 0.5750 [-0.4079; 0.9406] 0.6
## Rindermann & Ackermann2020 B 0.7640 [ 0.1840; 0.9494] 1.1
## Lim1988 0.9040 [ 0.7767; 0.9603] 3.6
## Stone1992 0.9900 [ 0.9596; 0.9976] 1.5
## Byrd & Buckhalt1991 0.9900 [ 0.9273; 0.9987] 0.7
## Tirre & Field2002 A 0.9900 [ 0.9728; 0.9963] 0.0
## Tirre & Field2002 B 0.9900 [ 0.9732; 0.9963] 0.0
## Tirre & Field2002 C 0.9730 [ 0.9109; 0.9920] 2.0
## Wothke et al.1991 0.9900 [ 0.9695; 0.9967] 0.0
## Williamson1969 0.9900 [ 0.9680; 0.9969] 0.0
## Kettner1976 A 0.9510 [ 0.8665; 0.9825] 2.7
## Kettner1976 B 0.9900 [ 0.9718; 0.9965] 0.0
## Palmer et al.1990 0.9750 [ 0.9360; 0.9904] 3.3
## Deary et al.1989 0.6100 [ 0.0120; 0.8866] 0.0
## Kranzler & Jensen1991 0.3680 [-0.3382; 0.8091] 0.0
## Luo, Thompson & Detterman2003 0.8700 [ 0.7141; 0.9437] 3.9
## Carey1992 0.8900 [ 0.7889; 0.9442] 6.0
## Abrahams et al.1994 0.8610 [ 0.7979; 0.9054] 0.0
## Wolfe et al.1995 0.8660 [ 0.8084; 0.9072] 0.0
## Naglieri & Jensen1987 0.9900 [ 0.9661; 0.9971] 2.0
## Engelhardt2018 0.9800 [ 0.9569; 0.9908] 5.0
## Stauffer, Ree & Carretta1996 0.9940 [ 0.9832; 0.9979] 0.0
## Keith, Kranzler & Flanagan2001 0.9900 [ 0.9647; 0.9972] 1.9
## Deary et al.2004 0.9200 [ 0.6052; 0.9860] 1.0
## Acton & Schroeder2001 0.6780 [ 0.4203; 0.8345] 0.0
## Meyer et al.2010 0.7800 [ 0.6085; 0.8819] 0.0
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 0.0
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 0.0
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 0.0
## Deary et al.2007 0.8100 [ 0.7361; 0.8648] 0.0
## Quiroga et al.2019 0.8320 [ 0.4818; 0.9530] 1.7
## Floyd et al.2013 0.9700 [ 0.9061; 0.9906] 2.2
## Floyd et al.2013 0.9500 [ 0.8297; 0.9860] 1.8
## Floyd et al.2013 0.9900 [ 0.9625; 0.9974] 1.7
## Floyd et al.2013 0.8900 [ 0.6162; 0.9719] 1.5
## Floyd et al.2013 0.9200 [ 0.6619; 0.9831] 1.2
## Peters, Kyngdon & Stillwell2021 0.7200 [ 0.2257; 0.9195] 0.0
## Reynolds et al.2015 0.8400 [ 0.5355; 0.9512] 1.9
## Valerius & Sparfeldt2014 A 0.9900 [ 0.9765; 0.9958] 0.0
## Valerius & Sparfeldt2014 B 0.9200 [ 0.8205; 0.9654] 4.1
## Valerius & Sparfeldt2014 C 0.9500 [ 0.8856; 0.9786] 4.1
## Quiroga et al.2015 0.9300 [ 0.7855; 0.9783] 2.1
## Swagerman et al.2016 0.9900 [ 0.9702; 0.9967] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.8900 [ 0.7515; 0.9534] 3.8
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.8100 [ 0.5926; 0.9174] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.7300 [ 0.4489; 0.8796] 0.0
## Johnson, te Nijenhuis & Bouchard2008 0.9300 [ 0.8376; 0.9707] 3.8
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 0.0
## Floyd et al.2010 0.9900 [ 0.9567; 0.9977] 1.4
## Kaufman et al.2012 0.8600 [ 0.7650; 0.9184] 0.0
## Kaufman et al.2012 0.8000 [ 0.6963; 0.8710] 0.0
## Salthouse2013 0.9100 [ 0.6399; 0.9800] 1.3
## Flanagan2000 0.8300 [ 0.5115; 0.9480] 1.9
## Floyd et al.2007 0.7770 [ 0.6472; 0.8630] 0.0
## Parkin & Beaujean2012 0.9100 [ 0.7984; 0.9612] 4.0
## Vanderwood et al.2002 0.9440 [ 0.9070; 0.9665] 10.9
## Freis et al.2020 0.7040 [ 0.5912; 0.7898] 0.0
## Buczylowska, Petermann & Daseking2020 0.9990 [ 0.9968; 0.9997] 0.0
## Wang et al.2021 0.5960 [ 0.1125; 0.8513] 0.0
## Snow et al.1977 0.9900 [ 0.9732; 0.9963] 0.0
## McGrew & Woodcock2001 A 0.9900 [ 0.9641; 0.9972] 1.8
## McGrew & Woodcock2001 B 0.9900 [ 0.9641; 0.9972] 1.8
## McGrew & Woodcock2001 C 0.9590 [ 0.8584; 0.9886] 1.8
## McGrew & Woodcock2001 D 0.9900 [ 0.9640; 0.9973] 1.8
## Woodcock1978 0.9790 [ 0.9290; 0.9939] 1.9
## Undheim1976 0.9900 [ 0.9636; 0.9973] 1.8
## Grigorenko et al.2004 0.6810 [ 0.2827; 0.8790] 0.0
## %W(random) exclude
## DeVries & Kohlberg1974 1.6
## Devries 1974 A 1.7
## Devries1974 B 2.0
## Hathaway1972 A 1.8
## Hathaway1972 B 1.8
## Rindermann & Ackermann2020 A 1.5
## Rindermann & Ackermann2020 B 2.0
## Lim1988 2.8
## Stone1992 2.2
## Byrd & Buckhalt1991 1.6
## Tirre & Field2002 A 0.0 *
## Tirre & Field2002 B 0.0 *
## Tirre & Field2002 C 2.5
## Wothke et al.1991 0.0 *
## Williamson1969 0.0 *
## Kettner1976 A 2.7
## Kettner1976 B 0.0 *
## Palmer et al.1990 2.8
## Deary et al.1989 0.0 *
## Kranzler & Jensen1991 0.0 *
## Luo, Thompson & Detterman2003 2.9
## Carey1992 3.1
## Abrahams et al.1994 0.0 *
## Wolfe et al.1995 0.0 *
## Naglieri & Jensen1987 2.4
## Engelhardt2018 3.0
## Stauffer, Ree & Carretta1996 0.0 *
## Keith, Kranzler & Flanagan2001 2.4
## Deary et al.2004 1.8
## Acton & Schroeder2001 0.0 *
## Meyer et al.2010 0.0 *
## Johnson et al.2004 0.0 *
## Johnson et al.2004 0.0 *
## Johnson et al.2004 0.0 *
## Deary et al.2007 0.0 *
## Quiroga et al.2019 2.3
## Floyd et al.2013 2.5
## Floyd et al.2013 2.4
## Floyd et al.2013 2.3
## Floyd et al.2013 2.2
## Floyd et al.2013 2.0
## Peters, Kyngdon & Stillwell2021 0.0 *
## Reynolds et al.2015 2.4
## Valerius & Sparfeldt2014 A 0.0 *
## Valerius & Sparfeldt2014 B 2.9
## Valerius & Sparfeldt2014 C 2.9
## Quiroga et al.2015 2.5
## Swagerman et al.2016 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 2.9
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Johnson, te Nijenhuis & Bouchard2008 2.9
## Johnson, te Nijenhuis & Bouchard2008 0.0 *
## Floyd et al.2010 2.2
## Kaufman et al.2012 0.0 *
## Kaufman et al.2012 0.0 *
## Salthouse2013 2.1
## Flanagan2000 2.4
## Floyd et al.2007 0.0 *
## Parkin & Beaujean2012 2.9
## Vanderwood et al.2002 3.3
## Freis et al.2020 0.0 *
## Buczylowska, Petermann & Daseking2020 0.0 *
## Wang et al.2021 0.0 *
## Snow et al.1977 0.0 *
## McGrew & Woodcock2001 A 2.4
## McGrew & Woodcock2001 B 2.4
## McGrew & Woodcock2001 C 2.4
## McGrew & Woodcock2001 D 2.4
## Woodcock1978 2.4
## Undheim1976 2.4
## Grigorenko et al.2004 0.0 *
##
## Number of studies combined: k = 42
##
## COR 95%-CI z p-value
## Fixed effect model 0.9490 [0.9397; 0.9570] 41.16 0
## Random effects model 0.9512 [0.9316; 0.9654] 20.69 < 0.0001
##
## Quantifying heterogeneity:
## tau^2 = 0.2258 [0.0905; 0.3880]; tau = 0.4752 [0.3008; 0.6229];
## I^2 = 64.5% [50.7%; 74.4%]; H = 1.68 [1.42; 1.98]
##
## Test of heterogeneity:
## Q d.f. p-value
## 115.35 41 < 0.0001
##
## Details on meta-analytical method:
## - Inverse variance method
## - Sidik-Jonkman estimator for tau^2
## - Q-profile method for confidence interval of tau^2 and tau
## - Fisher's z transformation of correlationsFOZ = find.outliers(ZCORRMA); FOZ; z2r(FOZ$m$b)## Identified outliers (ML)
## -------------------------
## "2", "3", "4", "6", "7", "8", "9", "10", "11", "12", "14", "15", "17", "18", "19", "20", "21", "22", "23", "24", "25", "26", "27", "28", "30", "31", "32", "33", "34", "35", "36", "39", "40", "42", "43", "44", "45", "48", "49", "50", "51", "52", "53", "54", "55", "56", "58", "59", "60", "61", "63", "64", "65", "67", "68", "69", "70", "71", "72", "74", "75", "76", "77"
##
## Results with outliers removed
## -----------------------------
##
## Random-Effects Model (k = 14; tau^2 estimator: ML)
##
## tau^2 (estimated amount of total heterogeneity): 0.0278 (SE = 0.0132)
## tau (square root of estimated tau^2 value): 0.1668
## I^2 (total heterogeneity / total variability): 89.11%
## H^2 (total variability / sampling variability): 9.18
##
## Test for Heterogeneity:
## Q(df = 13) = 74.5556, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 1.7733 0.0502 35.3166 <.0001 1.6749 1.8717 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1## [,1]
## intrcpt 0.9439691infZC <- InfluenceAnalysis(ZCOR, random = T); infZC; plot(infZC, "influence")## [===========================================================================] DONE## Leave-One-Out Analysis (Sorted by I2)
## -----------------------------------
## Effect LLCI ULCI I2
## Omitting Freis et al.2020 1.891 1.723 2.059 0.889
## Omitting Buczylowska, Petermann & Daseking2020 1.850 1.688 2.013 0.891
## Omitting Deary et al.2007 1.887 1.718 2.056 0.894
## Omitting Stauffer, Ree & Carretta1996 1.862 1.694 2.029 0.895
## Omitting Kaufman et al.2012.1 1.887 1.718 2.056 0.895
## Omitting Floyd et al.2007 1.888 1.719 2.057 0.895
## Omitting Tirre & Field2002 A 1.865 1.696 2.034 0.896
## Omitting Tirre & Field2002 B 1.865 1.696 2.034 0.896
## Omitting Acton & Schroeder2001 1.891 1.723 2.058 0.896
## Omitting Johnson et al.2004 1.865 1.696 2.034 0.896
## Omitting Johnson et al.2004.1 1.865 1.696 2.034 0.896
## Omitting Johnson et al.2004.2 1.865 1.696 2.034 0.896
## Omitting Valerius & Sparfeldt2014 A 1.865 1.696 2.033 0.896
## Omitting Johnson, te Nijenhuis & Bouchard2008 1.865 1.696 2.034 0.896
## Omitting Johnson, te Nijenhuis & Bouchard2008.2 1.865 1.696 2.034 0.896
## Omitting Johnson, te Nijenhuis & Bouchard2008.3 1.865 1.696 2.034 0.896
## Omitting Johnson, te Nijenhuis & Bouchard2008.5 1.865 1.696 2.034 0.896
## Omitting Johnson, te Nijenhuis & Bouchard2008.6 1.865 1.696 2.034 0.896
## Omitting Johnson, te Nijenhuis & Bouchard2008.9 1.865 1.696 2.034 0.896
## Omitting Snow et al.1977 1.865 1.696 2.034 0.896
## Omitting Stone1992 1.866 1.697 2.035 0.897
## Omitting Wothke et al.1991 1.865 1.696 2.034 0.897
## Omitting Williamson1969 1.865 1.696 2.034 0.897
## Omitting Kettner1976 B 1.865 1.696 2.034 0.897
## Omitting Deary et al.1989 1.890 1.722 2.057 0.897
## Omitting Kranzler & Jensen1991 1.893 1.727 2.059 0.897
## Omitting Abrahams et al.1994 1.884 1.715 2.054 0.897
## Omitting Wolfe et al.1995 1.884 1.714 2.053 0.897
## Omitting Naglieri & Jensen1987 1.866 1.697 2.034 0.897
## Omitting Engelhardt2018 1.869 1.700 2.039 0.897
## Omitting Keith, Kranzler & Flanagan2001 1.866 1.697 2.035 0.897
## Omitting Meyer et al.2010 1.888 1.719 2.056 0.897
## Omitting Floyd et al.2013.2 1.866 1.697 2.035 0.897
## Omitting Swagerman et al.2016 1.865 1.696 2.034 0.897
## Omitting Johnson, te Nijenhuis & Bouchard2008.7 1.889 1.720 2.057 0.897
## Omitting Wang et al.2021 1.891 1.724 2.058 0.897
## Omitting McGrew & Woodcock2001 A 1.866 1.697 2.035 0.897
## Omitting McGrew & Woodcock2001 B 1.866 1.697 2.035 0.897
## Omitting McGrew & Woodcock2001 D 1.866 1.697 2.035 0.897
## Omitting Undheim1976 1.866 1.697 2.035 0.897
## Omitting Grigorenko et al.2004 1.889 1.722 2.057 0.897
## Omitting DeVries & Kohlberg1974 1.879 1.710 2.049 0.898
## Omitting Devries 1974 A 1.883 1.714 2.052 0.898
## Omitting Devries1974 B 1.885 1.716 2.053 0.898
## Omitting Hathaway1972 A 1.880 1.711 2.050 0.898
## Omitting Hathaway1972 B 1.878 1.708 2.048 0.898
## Omitting Rindermann & Ackermann2020 A 1.887 1.719 2.054 0.898
## Omitting Rindermann & Ackermann2020 B 1.885 1.717 2.054 0.898
## Omitting Lim1988 1.881 1.711 2.051 0.898
## Omitting Byrd & Buckhalt1991 1.868 1.699 2.037 0.898
## Omitting Tirre & Field2002 C 1.872 1.702 2.042 0.898
## Omitting Kettner1976 A 1.876 1.706 2.046 0.898
## Omitting Palmer et al.1990 1.871 1.701 2.041 0.898
## Omitting Luo, Thompson & Detterman2003 1.883 1.713 2.053 0.898
## Omitting Carey1992 1.882 1.712 2.052 0.898
## Omitting Deary et al.2004 1.879 1.709 2.048 0.898
## Omitting Quiroga et al.2019 1.884 1.715 2.053 0.898
## Omitting Floyd et al.2013 1.873 1.703 2.043 0.898
## Omitting Floyd et al.2013.1 1.876 1.706 2.046 0.898
## Omitting Floyd et al.2013.3 1.881 1.711 2.051 0.898
## Omitting Floyd et al.2013.4 1.879 1.709 2.049 0.898
## Omitting Peters, Kyngdon & Stillwell2021 1.887 1.719 2.056 0.898
## Omitting Reynolds et al.2015 1.884 1.715 2.053 0.898
## Omitting Valerius & Sparfeldt2014 B 1.879 1.709 2.049 0.898
## Omitting Valerius & Sparfeldt2014 C 1.876 1.706 2.046 0.898
## Omitting Quiroga et al.2015 1.878 1.708 2.048 0.898
## Omitting Johnson, te Nijenhuis & Bouchard2008.1 1.882 1.712 2.052 0.898
## Omitting Johnson, te Nijenhuis & Bouchard2008.4 1.886 1.717 2.055 0.898
## Omitting Johnson, te Nijenhuis & Bouchard2008.8 1.878 1.708 2.049 0.898
## Omitting Floyd et al.2010 1.866 1.697 2.035 0.898
## Omitting Kaufman et al.2012 1.884 1.715 2.054 0.898
## Omitting Salthouse2013 1.880 1.710 2.049 0.898
## Omitting Flanagan2000 1.884 1.715 2.053 0.898
## Omitting Parkin & Beaujean2012 1.880 1.710 2.050 0.898
## Omitting Vanderwood et al.2002 1.877 1.707 2.047 0.898
## Omitting McGrew & Woodcock2001 C 1.875 1.705 2.045 0.898
## Omitting Woodcock1978 1.870 1.701 2.040 0.898
##
##
## Influence Diagnostics
## -------------------
## rstudent dffits cook.d cov.r
## Omitting DeVries & Kohlberg1974 -0.451 -0.045 0.002 1.020
## Omitting Devries 1974 A -0.876 -0.089 0.008 1.012
## Omitting Devries1974 B -1.010 -0.108 0.012 1.010
## Omitting Hathaway1972 A -0.530 -0.055 0.003 1.020
## Omitting Hathaway1972 B -0.310 -0.032 0.001 1.022
## Omitting Rindermann & Ackermann2020 A -1.388 -0.134 0.018 0.995
## Omitting Rindermann & Ackermann2020 B -1.082 -0.115 0.013 1.008
## Omitting Lim1988 -0.524 -0.062 0.004 1.023
## Omitting Stone1992 0.994 0.110 0.012 1.012
## Omitting Byrd & Buckhalt1991 0.898 0.090 0.008 1.012
## Omitting Tirre & Field2002 A 1.052 0.123 0.015 1.012
## Omitting Tirre & Field2002 B 1.054 0.123 0.015 1.012
## Omitting Tirre & Field2002 C 0.356 0.040 0.002 1.024
## Omitting Wothke et al.1991 1.036 0.119 0.014 1.012
## Omitting Williamson1969 1.029 0.118 0.014 1.012
## Omitting Kettner1976 A -0.045 -0.005 0.000 1.027
## Omitting Kettner1976 B 1.047 0.122 0.015 1.012
## Omitting Palmer et al.1990 0.422 0.050 0.002 1.025
## Omitting Deary et al.1989 -1.520 -0.168 0.028 0.995
## Omitting Kranzler & Jensen1991 -1.932 -0.210 0.043 0.976
## Omitting Luo, Thompson & Detterman2003 -0.748 -0.089 0.008 1.020
## Omitting Carey1992 -0.636 -0.077 0.006 1.022
## Omitting Abrahams et al.1994 -0.830 -0.102 0.010 1.019
## Omitting Wolfe et al.1995 -0.802 -0.099 0.010 1.020
## Omitting Naglieri & Jensen1987 1.021 0.116 0.013 1.012
## Omitting Engelhardt2018 0.587 0.070 0.005 1.023
## Omitting Stauffer, Ree & Carretta1996 1.404 0.163 0.026 1.001
## Omitting Keith, Kranzler & Flanagan2001 1.015 0.114 0.013 1.012
## Omitting Deary et al.2004 -0.346 -0.036 0.001 1.022
## Omitting Acton & Schroeder2001 -1.483 -0.178 0.031 0.999
## Omitting Meyer et al.2010 -1.174 -0.142 0.020 1.010
## Omitting Johnson et al.2004 1.062 0.125 0.016 1.012
## Omitting Johnson et al.2004.1 1.062 0.125 0.016 1.012
## Omitting Johnson et al.2004.2 1.062 0.125 0.016 1.012
## Omitting Deary et al.2007 -1.080 -0.133 0.018 1.013
## Omitting Quiroga et al.2019 -0.886 -0.099 0.010 1.015
## Omitting Floyd et al.2013 0.288 0.033 0.001 1.025
## Omitting Floyd et al.2013.1 -0.057 -0.006 0.000 1.026
## Omitting Floyd et al.2013.2 1.006 0.112 0.013 1.012
## Omitting Floyd et al.2013.3 -0.583 -0.064 0.004 1.021
## Omitting Floyd et al.2013.4 -0.357 -0.038 0.001 1.023
## Omitting Peters, Kyngdon & Stillwell2021 -1.263 -0.141 0.020 1.004
## Omitting Reynolds et al.2015 -0.863 -0.098 0.010 1.016
## Omitting Valerius & Sparfeldt2014 A 1.070 0.127 0.016 1.012
## Omitting Valerius & Sparfeldt2014 B -0.395 -0.047 0.002 1.025
## Omitting Valerius & Sparfeldt2014 C -0.060 -0.007 0.000 1.027
## Omitting Quiroga et al.2015 -0.287 -0.033 0.001 1.025
## Omitting Swagerman et al.2016 1.040 0.120 0.014 1.012
## Omitting Johnson, te Nijenhuis & Bouchard2008 1.067 0.126 0.016 1.012
## Omitting Johnson, te Nijenhuis & Bouchard2008.1 -0.624 -0.074 0.005 1.022
## Omitting Johnson, te Nijenhuis & Bouchard2008.2 1.067 0.126 0.016 1.012
## Omitting Johnson, te Nijenhuis & Bouchard2008.3 1.067 0.126 0.016 1.012
## Omitting Johnson, te Nijenhuis & Bouchard2008.4 -1.034 -0.122 0.015 1.013
## Omitting Johnson, te Nijenhuis & Bouchard2008.5 1.067 0.126 0.016 1.012
## Omitting Johnson, te Nijenhuis & Bouchard2008.6 1.067 0.126 0.016 1.012
## Omitting Johnson, te Nijenhuis & Bouchard2008.7 -1.314 -0.156 0.024 1.005
## Omitting Johnson, te Nijenhuis & Bouchard2008.8 -0.298 -0.035 0.001 1.026
## Omitting Johnson, te Nijenhuis & Bouchard2008.9 1.067 0.126 0.016 1.012
## Omitting Floyd et al.2010 0.983 0.107 0.012 1.012
## Omitting Kaufman et al.2012 -0.827 -0.101 0.010 1.019
## Omitting Kaufman et al.2012.1 -1.115 -0.137 0.019 1.012
## Omitting Salthouse2013 -0.438 -0.047 0.002 1.022
## Omitting Flanagan2000 -0.907 -0.103 0.011 1.015
## Omitting Floyd et al.2007 -1.199 -0.147 0.021 1.009
## Omitting Parkin & Beaujean2012 -0.479 -0.057 0.003 1.024
## Omitting Vanderwood et al.2002 -0.144 -0.018 0.000 1.028
## Omitting Freis et al.2020 -1.452 -0.179 0.032 1.001
## Omitting Buczylowska, Petermann & Daseking2020 2.670 0.305 0.086 0.938
## Omitting Wang et al.2021 -1.607 -0.184 0.033 0.992
## Omitting Snow et al.1977 1.054 0.123 0.015 1.012
## Omitting McGrew & Woodcock2001 A 1.012 0.114 0.013 1.012
## Omitting McGrew & Woodcock2001 B 1.012 0.114 0.013 1.012
## Omitting McGrew & Woodcock2001 C 0.075 0.009 0.000 1.026
## Omitting McGrew & Woodcock2001 D 1.012 0.114 0.013 1.012
## Omitting Woodcock1978 0.523 0.059 0.004 1.022
## Omitting Undheim1976 1.010 0.113 0.013 1.012
## Omitting Grigorenko et al.2004 -1.420 -0.164 0.027 1.000
## QE.del hat weight infl
## Omitting DeVries & Kohlberg1974 738.570 0.010 0.993
## Omitting Devries 1974 A 737.551 0.010 1.012
## Omitting Devries1974 B 736.601 0.011 1.130
## Omitting Hathaway1972 A 738.450 0.010 1.044
## Omitting Hathaway1972 B 738.675 0.010 1.044
## Omitting Rindermann & Ackermann2020 A 735.401 0.009 0.932
## Omitting Rindermann & Ackermann2020 B 736.229 0.011 1.121
## Omitting Lim1988 738.170 0.014 1.374
## Omitting Stone1992 731.124 0.012 1.205
## Omitting Byrd & Buckhalt1991 734.991 0.010 0.986
## Omitting Tirre & Field2002 A 723.794 0.013 1.345
## Omitting Tirre & Field2002 B 723.335 0.013 1.350
## Omitting Tirre & Field2002 C 736.252 0.013 1.271
## Omitting Wothke et al.1991 726.741 0.013 1.306
## Omitting Williamson1969 727.753 0.013 1.289
## Omitting Kettner1976 A 738.213 0.013 1.332
## Omitting Kettner1976 B 724.895 0.013 1.332
## Omitting Palmer et al.1990 734.039 0.014 1.360
## Omitting Deary et al.1989 731.497 0.012 1.211
## Omitting Kranzler & Jensen1991 727.195 0.012 1.181
## Omitting Luo, Thompson & Detterman2003 736.515 0.014 1.386
## Omitting Carey1992 736.908 0.014 1.433
## Omitting Abrahams et al.1994 725.814 0.015 1.496
## Omitting Wolfe et al.1995 725.882 0.015 1.499
## Omitting Naglieri & Jensen1987 728.768 0.013 1.269
## Omitting Engelhardt2018 728.228 0.014 1.413
## Omitting Stauffer, Ree & Carretta1996 716.431 0.013 1.335
## Omitting Keith, Kranzler & Flanagan2001 729.424 0.013 1.253
## Omitting Deary et al.2004 738.660 0.011 1.072
## Omitting Acton & Schroeder2001 719.565 0.014 1.420
## Omitting Meyer et al.2010 725.770 0.014 1.440
## Omitting Johnson et al.2004 721.065 0.014 1.370
## Omitting Johnson et al.2004.1 721.065 0.014 1.370
## Omitting Johnson et al.2004.2 721.065 0.014 1.370
## Omitting Deary et al.2007 704.380 0.015 1.502
## Omitting Quiroga et al.2019 736.818 0.012 1.231
## Omitting Floyd et al.2013 736.584 0.013 1.289
## Omitting Floyd et al.2013.1 738.409 0.012 1.249
## Omitting Floyd et al.2013.2 730.243 0.012 1.232
## Omitting Floyd et al.2013.3 738.242 0.012 1.206
## Omitting Floyd et al.2013.4 738.653 0.011 1.141
## Omitting Peters, Kyngdon & Stillwell2021 733.931 0.012 1.225
## Omitting Reynolds et al.2015 736.771 0.013 1.263
## Omitting Valerius & Sparfeldt2014 A 718.243 0.014 1.390
## Omitting Valerius & Sparfeldt2014 B 738.579 0.014 1.390
## Omitting Valerius & Sparfeldt2014 C 738.063 0.014 1.390
## Omitting Quiroga et al.2015 738.684 0.013 1.281
## Omitting Swagerman et al.2016 726.228 0.013 1.314
## Omitting Johnson, te Nijenhuis & Bouchard2008 719.589 0.014 1.381
## Omitting Johnson, te Nijenhuis & Bouchard2008.1 737.575 0.014 1.381
## Omitting Johnson, te Nijenhuis & Bouchard2008.2 719.589 0.014 1.381
## Omitting Johnson, te Nijenhuis & Bouchard2008.3 719.589 0.014 1.381
## Omitting Johnson, te Nijenhuis & Bouchard2008.4 733.120 0.014 1.381
## Omitting Johnson, te Nijenhuis & Bouchard2008.5 719.589 0.014 1.381
## Omitting Johnson, te Nijenhuis & Bouchard2008.6 719.589 0.014 1.381
## Omitting Johnson, te Nijenhuis & Bouchard2008.7 728.206 0.014 1.381
## Omitting Johnson, te Nijenhuis & Bouchard2008.8 738.684 0.014 1.381
## Omitting Johnson, te Nijenhuis & Bouchard2008.9 719.589 0.014 1.381
## Omitting Floyd et al.2010 731.835 0.012 1.179
## Omitting Kaufman et al.2012 732.147 0.015 1.465
## Omitting Kaufman et al.2012.1 716.461 0.015 1.484
## Omitting Salthouse2013 738.570 0.012 1.158
## Omitting Flanagan2000 736.471 0.013 1.263
## Omitting Floyd et al.2007 717.144 0.015 1.473
## Omitting Parkin & Beaujean2012 738.322 0.014 1.388
## Omitting Vanderwood et al.2002 737.938 0.015 1.475
## Omitting Freis et al.2020 672.686 0.015 1.499
## Omitting Buczylowska, Petermann & Daseking2020 686.449 0.013 1.292
## Omitting Wang et al.2021 727.562 0.013 1.298
## Omitting Snow et al.1977 723.335 0.013 1.350
## Omitting McGrew & Woodcock2001 A 729.664 0.012 1.247
## Omitting McGrew & Woodcock2001 B 729.664 0.012 1.247
## Omitting McGrew & Woodcock2001 C 737.991 0.012 1.247
## Omitting McGrew & Woodcock2001 D 729.704 0.012 1.246
## Omitting Woodcock1978 734.930 0.013 1.264
## Omitting Undheim1976 729.867 0.012 1.242
## Omitting Grigorenko et al.2004 729.570 0.013 1.321
##
##
## Baujat Diagnostics (sorted by Heterogeneity Contribution)
## -------------------------------------------------------
## HetContrib InfluenceEffectSize
## Omitting Freis et al.2020 61.714 4.283
## Omitting Buczylowska, Petermann & Daseking2020 51.852 0.383
## Omitting Deary et al.2007 31.815 2.489
## Omitting Stauffer, Ree & Carretta1996 22.047 0.205
## Omitting Kaufman et al.2012.1 21.252 0.971
## Omitting Floyd et al.2007 20.790 0.749
## Omitting Valerius & Sparfeldt2014 A 20.167 0.274
## Omitting Johnson, te Nijenhuis & Bouchard2008 18.856 0.239
## Omitting Johnson, te Nijenhuis & Bouchard2008.2 18.856 0.239
## Omitting Johnson, te Nijenhuis & Bouchard2008.3 18.856 0.239
## Omitting Johnson, te Nijenhuis & Bouchard2008.5 18.856 0.239
## Omitting Johnson, te Nijenhuis & Bouchard2008.6 18.856 0.239
## Omitting Johnson, te Nijenhuis & Bouchard2008.9 18.856 0.239
## Omitting Acton & Schroeder2001 18.785 0.334
## Omitting Johnson et al.2004 17.415 0.204
## Omitting Johnson et al.2004.1 17.415 0.204
## Omitting Johnson et al.2004.2 17.415 0.204
## Omitting Tirre & Field2002 B 15.195 0.155
## Omitting Snow et al.1977 15.195 0.155
## Omitting Tirre & Field2002 A 14.744 0.146
## Omitting Kettner1976 B 13.664 0.125
## Omitting Meyer et al.2010 12.634 0.280
## Omitting Swagerman et al.2016 12.354 0.102
## Omitting Abrahams et al.1994 12.103 0.767
## Omitting Wolfe et al.1995 11.959 0.843
## Omitting Wothke et al.1991 11.849 0.094
## Omitting Kranzler & Jensen1991 11.436 0.052
## Omitting Wang et al.2021 11.038 0.084
## Omitting Williamson1969 10.852 0.079
## Omitting Johnson, te Nijenhuis & Bouchard2008.7 10.347 0.131
## Omitting Engelhardt2018 10.285 0.171
## Omitting Naglieri & Jensen1987 9.851 0.065
## Omitting Keith, Kranzler & Flanagan2001 9.204 0.057
## Omitting Grigorenko et al.2004 9.036 0.078
## Omitting McGrew & Woodcock2001 A 8.967 0.054
## Omitting McGrew & Woodcock2001 B 8.967 0.054
## Omitting McGrew & Woodcock2001 D 8.927 0.053
## Omitting Undheim1976 8.766 0.051
## Omitting Floyd et al.2013.2 8.394 0.047
## Omitting Stone1992 7.523 0.038
## Omitting Deary et al.1989 7.150 0.037
## Omitting Floyd et al.2010 6.818 0.031
## Omitting Kaufman et al.2012 6.337 0.200
## Omitting Johnson, te Nijenhuis & Bouchard2008.4 5.495 0.070
## Omitting Peters, Kyngdon & Stillwell2021 4.728 0.026
## Omitting Palmer et al.1990 4.595 0.050
## Omitting Woodcock1978 3.730 0.024
## Omitting Byrd & Buckhalt1991 3.684 0.009
## Omitting Rindermann & Ackermann2020 A 3.276 0.007
## Omitting Rindermann & Ackermann2020 B 2.446 0.009
## Omitting Tirre & Field2002 C 2.416 0.016
## Omitting Flanagan2000 2.199 0.014
## Omitting Luo, Thompson & Detterman2003 2.141 0.028
## Omitting Floyd et al.2013 2.084 0.015
## Omitting Devries1974 B 2.075 0.008
## Omitting Reynolds et al.2015 1.901 0.012
## Omitting Quiroga et al.2019 1.856 0.010
## Omitting Carey1992 1.741 0.035
## Omitting Devries 1974 A 1.130 0.003
## Omitting Johnson, te Nijenhuis & Bouchard2008.1 1.095 0.014
## Omitting Vanderwood et al.2002 0.719 0.027
## Omitting McGrew & Woodcock2001 C 0.689 0.004
## Omitting Valerius & Sparfeldt2014 C 0.613 0.008
## Omitting Lim1988 0.508 0.006
## Omitting Kettner1976 A 0.467 0.004
## Omitting Floyd et al.2013.3 0.440 0.002
## Omitting Parkin & Beaujean2012 0.358 0.005
## Omitting Floyd et al.2013.1 0.274 0.002
## Omitting Hathaway1972 A 0.233 0.001
## Omitting DeVries & Kohlberg1974 0.113 0.000
## Omitting Salthouse2013 0.113 0.000
## Omitting Valerius & Sparfeldt2014 B 0.104 0.001
## Omitting Floyd et al.2013.4 0.031 0.000
## Omitting Deary et al.2004 0.024 0.000
## Omitting Hathaway1972 B 0.009 0.000
## Omitting Quiroga et al.2015 0.000 0.000
## Omitting Johnson, te Nijenhuis & Bouchard2008.8 0.000 0.000
plot(infZC, "baujat")
plot(infZC, "es")
radial(ZCORRMA)
baujat(ZCORRMA)
Because of the large influence of Freis et al. (2020) and Deary et al. (2007), a sensitivity analysis without them is justified. This is presented here:
JOGS <- JOG[-c(35, 67), ]; JOGSZCORS <- metacor(rG,
sqrt(n),
data = JOGS,
studlab = SY,
sm = "ZCOR",
method.tau = "SJ")
ZCORS## COR 95%-CI %W(fixed)
## DeVries & Kohlberg1974 0.9030 [ 0.4543; 0.9862] 0.3
## Devries 1974 A 0.8120 [ 0.1601; 0.9707] 0.3
## Devries1974 B 0.7880 [ 0.2528; 0.9539] 0.4
## Hathaway1972 A 0.8920 [ 0.4668; 0.9822] 0.3
## Hathaway1972 B 0.9240 [ 0.5980; 0.9877] 0.3
## Rindermann & Ackermann2020 A 0.5750 [-0.4079; 0.9406] 0.2
## Rindermann & Ackermann2020 B 0.7640 [ 0.1840; 0.9494] 0.4
## Lim1988 0.9040 [ 0.7767; 0.9603] 1.4
## Stone1992 0.9900 [ 0.9596; 0.9976] 0.6
## Byrd & Buckhalt1991 0.9900 [ 0.9273; 0.9987] 0.3
## Tirre & Field2002 A 0.9900 [ 0.9728; 0.9963] 1.1
## Tirre & Field2002 B 0.9900 [ 0.9732; 0.9963] 1.2
## Tirre & Field2002 C 0.9730 [ 0.9109; 0.9920] 0.8
## Wothke et al.1991 0.9900 [ 0.9695; 0.9967] 0.9
## Williamson1969 0.9900 [ 0.9680; 0.9969] 0.8
## Kettner1976 A 0.9510 [ 0.8665; 0.9825] 1.1
## Kettner1976 B 0.9900 [ 0.9718; 0.9965] 1.1
## Palmer et al.1990 0.9750 [ 0.9360; 0.9904] 1.3
## Deary et al.1989 0.6100 [ 0.0120; 0.8866] 0.6
## Kranzler & Jensen1991 0.3680 [-0.3382; 0.8091] 0.5
## Luo, Thompson & Detterman2003 0.8700 [ 0.7141; 0.9437] 1.5
## Carey1992 0.8900 [ 0.7889; 0.9442] 2.3
## Abrahams et al.1994 0.8610 [ 0.7979; 0.9054] 6.9
## Wolfe et al.1995 0.8660 [ 0.8084; 0.9072] 7.6
## Naglieri & Jensen1987 0.9900 [ 0.9661; 0.9971] 0.8
## Engelhardt2018 0.9800 [ 0.9569; 0.9908] 1.9
## Stauffer, Ree & Carretta1996 0.9940 [ 0.9832; 0.9979] 1.1
## Keith, Kranzler & Flanagan2001 0.9900 [ 0.9647; 0.9972] 0.7
## Deary et al.2004 0.9200 [ 0.6052; 0.9860] 0.4
## Acton & Schroeder2001 0.6780 [ 0.4203; 0.8345] 2.0
## Meyer et al.2010 0.7800 [ 0.6085; 0.8819] 2.5
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 1.3
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 1.3
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 1.3
## Quiroga et al.2019 0.8320 [ 0.4818; 0.9530] 0.6
## Floyd et al.2013 0.9700 [ 0.9061; 0.9906] 0.8
## Floyd et al.2013 0.9500 [ 0.8297; 0.9860] 0.7
## Floyd et al.2013 0.9900 [ 0.9625; 0.9974] 0.6
## Floyd et al.2013 0.8900 [ 0.6162; 0.9719] 0.6
## Floyd et al.2013 0.9200 [ 0.6619; 0.9831] 0.5
## Peters, Kyngdon & Stillwell2021 0.7200 [ 0.2257; 0.9195] 0.6
## Reynolds et al.2015 0.8400 [ 0.5355; 0.9512] 0.7
## Valerius & Sparfeldt2014 A 0.9900 [ 0.9765; 0.9958] 1.6
## Valerius & Sparfeldt2014 B 0.9200 [ 0.8205; 0.9654] 1.6
## Valerius & Sparfeldt2014 C 0.9500 [ 0.8856; 0.9786] 1.6
## Quiroga et al.2015 0.9300 [ 0.7855; 0.9783] 0.8
## Swagerman et al.2016 0.9900 [ 0.9702; 0.9967] 1.0
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.5
## Johnson, te Nijenhuis & Bouchard2008 0.8900 [ 0.7515; 0.9534] 1.5
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.5
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.5
## Johnson, te Nijenhuis & Bouchard2008 0.8100 [ 0.5926; 0.9174] 1.5
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.5
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.5
## Johnson, te Nijenhuis & Bouchard2008 0.7300 [ 0.4489; 0.8796] 1.5
## Johnson, te Nijenhuis & Bouchard2008 0.9300 [ 0.8376; 0.9707] 1.5
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.5
## Floyd et al.2010 0.9900 [ 0.9567; 0.9977] 0.5
## Kaufman et al.2012 0.8600 [ 0.7650; 0.9184] 3.5
## Kaufman et al.2012 0.8000 [ 0.6963; 0.8710] 5.1
## Salthouse2013 0.9100 [ 0.6399; 0.9800] 0.5
## Flanagan2000 0.8300 [ 0.5115; 0.9480] 0.7
## Floyd et al.2007 0.7770 [ 0.6472; 0.8630] 4.0
## Parkin & Beaujean2012 0.9100 [ 0.7984; 0.9612] 1.5
## Vanderwood et al.2002 0.9440 [ 0.9070; 0.9665] 4.2
## Buczylowska, Petermann & Daseking2020 0.9990 [ 0.9968; 0.9997] 0.8
## Wang et al.2021 0.5960 [ 0.1125; 0.8513] 0.9
## Snow et al.1977 0.9900 [ 0.9732; 0.9963] 1.2
## McGrew & Woodcock2001 A 0.9900 [ 0.9641; 0.9972] 0.7
## McGrew & Woodcock2001 B 0.9900 [ 0.9641; 0.9972] 0.7
## McGrew & Woodcock2001 C 0.9590 [ 0.8584; 0.9886] 0.7
## McGrew & Woodcock2001 D 0.9900 [ 0.9640; 0.9973] 0.7
## Woodcock1978 0.9790 [ 0.9290; 0.9939] 0.7
## Undheim1976 0.9900 [ 0.9636; 0.9973] 0.7
## Grigorenko et al.2004 0.6810 [ 0.2827; 0.8790] 1.0
## %W(random)
## DeVries & Kohlberg1974 1.0
## Devries 1974 A 1.0
## Devries1974 B 1.2
## Hathaway1972 A 1.1
## Hathaway1972 B 1.1
## Rindermann & Ackermann2020 A 1.0
## Rindermann & Ackermann2020 B 1.2
## Lim1988 1.4
## Stone1992 1.2
## Byrd & Buckhalt1991 1.0
## Tirre & Field2002 A 1.4
## Tirre & Field2002 B 1.4
## Tirre & Field2002 C 1.3
## Wothke et al.1991 1.3
## Williamson1969 1.3
## Kettner1976 A 1.4
## Kettner1976 B 1.4
## Palmer et al.1990 1.4
## Deary et al.1989 1.2
## Kranzler & Jensen1991 1.2
## Luo, Thompson & Detterman2003 1.4
## Carey1992 1.5
## Abrahams et al.1994 1.5
## Wolfe et al.1995 1.5
## Naglieri & Jensen1987 1.3
## Engelhardt2018 1.5
## Stauffer, Ree & Carretta1996 1.4
## Keith, Kranzler & Flanagan2001 1.3
## Deary et al.2004 1.1
## Acton & Schroeder2001 1.5
## Meyer et al.2010 1.5
## Johnson et al.2004 1.4
## Johnson et al.2004 1.4
## Johnson et al.2004 1.4
## Quiroga et al.2019 1.3
## Floyd et al.2013 1.3
## Floyd et al.2013 1.3
## Floyd et al.2013 1.3
## Floyd et al.2013 1.2
## Floyd et al.2013 1.2
## Peters, Kyngdon & Stillwell2021 1.3
## Reynolds et al.2015 1.3
## Valerius & Sparfeldt2014 A 1.4
## Valerius & Sparfeldt2014 B 1.4
## Valerius & Sparfeldt2014 C 1.4
## Quiroga et al.2015 1.3
## Swagerman et al.2016 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Johnson, te Nijenhuis & Bouchard2008 1.4
## Floyd et al.2010 1.2
## Kaufman et al.2012 1.5
## Kaufman et al.2012 1.5
## Salthouse2013 1.2
## Flanagan2000 1.3
## Floyd et al.2007 1.5
## Parkin & Beaujean2012 1.4
## Vanderwood et al.2002 1.5
## Buczylowska, Petermann & Daseking2020 1.3
## Wang et al.2021 1.3
## Snow et al.1977 1.4
## McGrew & Woodcock2001 A 1.3
## McGrew & Woodcock2001 B 1.3
## McGrew & Woodcock2001 C 1.3
## McGrew & Woodcock2001 D 1.3
## Woodcock1978 1.3
## Undheim1976 1.3
## Grigorenko et al.2004 1.4
##
## Number of studies combined: k = 75
##
## COR 95%-CI z p-value
## Fixed effect model 0.9429 [0.9366; 0.9486] 64.38 0
## Random effects model 0.9565 [0.9395; 0.9688] 22.05 < 0.0001
##
## Quantifying heterogeneity:
## tau^2 = 0.4705 [0.3185; 0.6886]; tau = 0.6859 [0.5644; 0.8298];
## I^2 = 88.3% [86.0%; 90.2%]; H = 2.92 [2.67; 3.19]
##
## Test of heterogeneity:
## Q d.f. p-value
## 630.78 74 < 0.0001
##
## Details on meta-analytical method:
## - Inverse variance method
## - Sidik-Jonkman estimator for tau^2
## - Q-profile method for confidence interval of tau^2 and tau
## - Fisher's z transformation of correlationsRMADATS <- escalc(measure = "ZCOR", ri = rG, ni = n, data = JOGS)
ZCORRMAS <- rma.uni(yi = yi, vi = vi, data = RMADAT, method = "ML"); ZCORRMAS; z2r(ZCORRMAS$b)##
## Random-Effects Model (k = 77; tau^2 estimator: ML)
##
## tau^2 (estimated amount of total heterogeneity): 0.5441 (SE = 0.0886)
## tau (square root of estimated tau^2 value): 0.7376
## I^2 (total heterogeneity / total variability): 99.81%
## H^2 (total variability / sampling variability): 516.83
##
## Test for Heterogeneity:
## Q(df = 76) = 22413.7875, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 1.8725 0.0845 22.1587 <.0001 1.7069 2.0381 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1## [,1]
## intrcpt 0.9538215DG <- gosh(ZCORRMA)## Fitting 1000000 models (based on random subsets).##
|
| | 0%
|
| | 1%
|
|= | 1%
|
|= | 2%
|
|== | 2%
|
|== | 3%
|
|== | 4%
|
|=== | 4%
|
|=== | 5%
|
|==== | 5%
|
|==== | 6%
|
|===== | 6%
|
|===== | 7%
|
|===== | 8%
|
|====== | 8%
|
|====== | 9%
|
|======= | 9%
|
|======= | 10%
|
|======= | 11%
|
|======== | 11%
|
|======== | 12%
|
|========= | 12%
|
|========= | 13%
|
|========= | 14%
|
|========== | 14%
|
|========== | 15%
|
|=========== | 15%
|
|=========== | 16%
|
|============ | 16%
|
|============ | 17%
|
|============ | 18%
|
|============= | 18%
|
|============= | 19%
|
|============== | 19%
|
|============== | 20%
|
|============== | 21%
|
|=============== | 21%
|
|=============== | 22%
|
|================ | 22%
|
|================ | 23%
|
|================ | 24%
|
|================= | 24%
|
|================= | 25%
|
|================== | 25%
|
|================== | 26%
|
|=================== | 26%
|
|=================== | 27%
|
|=================== | 28%
|
|==================== | 28%
|
|==================== | 29%
|
|===================== | 29%
|
|===================== | 30%
|
|===================== | 31%
|
|====================== | 31%
|
|====================== | 32%
|
|======================= | 32%
|
|======================= | 33%
|
|======================= | 34%
|
|======================== | 34%
|
|======================== | 35%
|
|========================= | 35%
|
|========================= | 36%
|
|========================== | 36%
|
|========================== | 37%
|
|========================== | 38%
|
|=========================== | 38%
|
|=========================== | 39%
|
|============================ | 39%
|
|============================ | 40%
|
|============================ | 41%
|
|============================= | 41%
|
|============================= | 42%
|
|============================== | 42%
|
|============================== | 43%
|
|============================== | 44%
|
|=============================== | 44%
|
|=============================== | 45%
|
|================================ | 45%
|
|================================ | 46%
|
|================================= | 46%
|
|================================= | 47%
|
|================================= | 48%
|
|================================== | 48%
|
|================================== | 49%
|
|=================================== | 49%
|
|=================================== | 50%
|
|=================================== | 51%
|
|==================================== | 51%
|
|==================================== | 52%
|
|===================================== | 52%
|
|===================================== | 53%
|
|===================================== | 54%
|
|====================================== | 54%
|
|====================================== | 55%
|
|======================================= | 55%
|
|======================================= | 56%
|
|======================================== | 56%
|
|======================================== | 57%
|
|======================================== | 58%
|
|========================================= | 58%
|
|========================================= | 59%
|
|========================================== | 59%
|
|========================================== | 60%
|
|========================================== | 61%
|
|=========================================== | 61%
|
|=========================================== | 62%
|
|============================================ | 62%
|
|============================================ | 63%
|
|============================================ | 64%
|
|============================================= | 64%
|
|============================================= | 65%
|
|============================================== | 65%
|
|============================================== | 66%
|
|=============================================== | 66%
|
|=============================================== | 67%
|
|=============================================== | 68%
|
|================================================ | 68%
|
|================================================ | 69%
|
|================================================= | 69%
|
|================================================= | 70%
|
|================================================= | 71%
|
|================================================== | 71%
|
|================================================== | 72%
|
|=================================================== | 72%
|
|=================================================== | 73%
|
|=================================================== | 74%
|
|==================================================== | 74%
|
|==================================================== | 75%
|
|===================================================== | 75%
|
|===================================================== | 76%
|
|====================================================== | 76%
|
|====================================================== | 77%
|
|====================================================== | 78%
|
|======================================================= | 78%
|
|======================================================= | 79%
|
|======================================================== | 79%
|
|======================================================== | 80%
|
|======================================================== | 81%
|
|========================================================= | 81%
|
|========================================================= | 82%
|
|========================================================== | 82%
|
|========================================================== | 83%
|
|========================================================== | 84%
|
|=========================================================== | 84%
|
|=========================================================== | 85%
|
|============================================================ | 85%
|
|============================================================ | 86%
|
|============================================================= | 86%
|
|============================================================= | 87%
|
|============================================================= | 88%
|
|============================================================== | 88%
|
|============================================================== | 89%
|
|=============================================================== | 89%
|
|=============================================================== | 90%
|
|=============================================================== | 91%
|
|================================================================ | 91%
|
|================================================================ | 92%
|
|================================================================= | 92%
|
|================================================================= | 93%
|
|================================================================= | 94%
|
|================================================================== | 94%
|
|================================================================== | 95%
|
|=================================================================== | 95%
|
|=================================================================== | 96%
|
|==================================================================== | 96%
|
|==================================================================== | 97%
|
|==================================================================== | 98%
|
|===================================================================== | 98%
|
|===================================================================== | 99%
|
|======================================================================| 99%
|
|======================================================================| 100%plot(DG, alpha = 0.1, col = "orange")
gosh.diagnostics(DG)##
## Perform Clustering...
## |==========================================================================================| DONE## GOSH Diagnostics
## ================================
##
## - Number of K-means clusters detected: 3
## - Number of DBSCAN clusters detected: 12
## - Number of GMM clusters detected: 4
##
## Identification of potential outliers
## ---------------------------------
##
## - K-means: Study 23, Study 24, Study 35, Study 61, Study 67, Study 20, Study 68
## - DBSCAN: Study 23, Study 24, Study 35, Study 61, Study 66, Study 67, Study 8, Study 22, Study 60, Study 64, Study 17, Study 19, Study 27, Study 33, Study 39, Study 57, Study 65, Study 70, Study 76, Study 4, Study 40, Study 43, Study 73, Study 9, Study 68, Study 11, Study 29, Study 36, Study 74, Study 12, Study 30, Study 58, Study 6, Study 51, Study 59
## - Gaussian Mixture Model: Study 23, Study 24, Study 35, Study 61, Study 66, Study 67, Study 8, Study 22, Study 60, Study 64, Study 17, Study 19, Study 27, Study 33, Study 39, Study 57, Study 65, Study 70, Study 76, Study 4, Study 40, Study 43, Study 73, Study 9, Study 68, Study 11, Study 29, Study 36, Study 74, Study 12, Study 30, Study 58, Study 6, Study 51, Study 59Sensitivity analyses based on the GOSH results are as follows:
JOGKM <- JOG[-c(20, 68, 35, 67, 23, 24, 61), ]; JOGKMJOGDB <- JOG[-c(23, 24, 35, 61, 67, 1, 5, 27, 31, 49, 63, 77, 19, 48, 66, 6, 9, 13, 20, 30, 46, 68, 70, 11, 15, 22, 40, 42, 51, 53, 56, 62, 71, 74, 3, 12, 14, 44, 72, 55, 2, 7, 34, 43, 16, 47), ]; JOGDBKMZ <- metacor(rG,
sqrt(n),
data = JOGKM,
studlab = SY,
sm = "ZCOR",
method.tau = "SJ"); KMZ## COR 95%-CI %W(fixed)
## DeVries & Kohlberg1974 0.9030 [ 0.4543; 0.9862] 0.4
## Devries 1974 A 0.8120 [ 0.1601; 0.9707] 0.4
## Devries1974 B 0.7880 [ 0.2528; 0.9539] 0.6
## Hathaway1972 A 0.8920 [ 0.4668; 0.9822] 0.4
## Hathaway1972 B 0.9240 [ 0.5980; 0.9877] 0.4
## Rindermann & Ackermann2020 A 0.5750 [-0.4079; 0.9406] 0.3
## Rindermann & Ackermann2020 B 0.7640 [ 0.1840; 0.9494] 0.5
## Lim1988 0.9040 [ 0.7767; 0.9603] 1.8
## Stone1992 0.9900 [ 0.9596; 0.9976] 0.7
## Byrd & Buckhalt1991 0.9900 [ 0.9273; 0.9987] 0.4
## Tirre & Field2002 A 0.9900 [ 0.9728; 0.9963] 1.4
## Tirre & Field2002 B 0.9900 [ 0.9732; 0.9963] 1.5
## Tirre & Field2002 C 0.9730 [ 0.9109; 0.9920] 1.0
## Wothke et al.1991 0.9900 [ 0.9695; 0.9967] 1.2
## Williamson1969 0.9900 [ 0.9680; 0.9969] 1.1
## Kettner1976 A 0.9510 [ 0.8665; 0.9825] 1.3
## Kettner1976 B 0.9900 [ 0.9718; 0.9965] 1.3
## Palmer et al.1990 0.9750 [ 0.9360; 0.9904] 1.6
## Deary et al.1989 0.6100 [ 0.0120; 0.8866] 0.8
## Luo, Thompson & Detterman2003 0.8700 [ 0.7141; 0.9437] 1.9
## Carey1992 0.8900 [ 0.7889; 0.9442] 2.9
## Naglieri & Jensen1987 0.9900 [ 0.9661; 0.9971] 1.0
## Engelhardt2018 0.9800 [ 0.9569; 0.9908] 2.4
## Stauffer, Ree & Carretta1996 0.9940 [ 0.9832; 0.9979] 1.4
## Keith, Kranzler & Flanagan2001 0.9900 [ 0.9647; 0.9972] 0.9
## Deary et al.2004 0.9200 [ 0.6052; 0.9860] 0.5
## Acton & Schroeder2001 0.6780 [ 0.4203; 0.8345] 2.6
## Meyer et al.2010 0.7800 [ 0.6085; 0.8819] 3.2
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 1.7
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 1.7
## Johnson et al.2004 0.9900 [ 0.9749; 0.9960] 1.7
## Quiroga et al.2019 0.8320 [ 0.4818; 0.9530] 0.8
## Floyd et al.2013 0.9700 [ 0.9061; 0.9906] 1.1
## Floyd et al.2013 0.9500 [ 0.8297; 0.9860] 0.9
## Floyd et al.2013 0.9900 [ 0.9625; 0.9974] 0.8
## Floyd et al.2013 0.8900 [ 0.6162; 0.9719] 0.7
## Floyd et al.2013 0.9200 [ 0.6619; 0.9831] 0.6
## Peters, Kyngdon & Stillwell2021 0.7200 [ 0.2257; 0.9195] 0.8
## Reynolds et al.2015 0.8400 [ 0.5355; 0.9512] 0.9
## Valerius & Sparfeldt2014 A 0.9900 [ 0.9765; 0.9958] 2.0
## Valerius & Sparfeldt2014 B 0.9200 [ 0.8205; 0.9654] 2.0
## Valerius & Sparfeldt2014 C 0.9500 [ 0.8856; 0.9786] 2.0
## Quiroga et al.2015 0.9300 [ 0.7855; 0.9783] 1.0
## Swagerman et al.2016 0.9900 [ 0.9702; 0.9967] 1.2
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.8
## Johnson, te Nijenhuis & Bouchard2008 0.8900 [ 0.7515; 0.9534] 1.8
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.8
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.8
## Johnson, te Nijenhuis & Bouchard2008 0.8100 [ 0.5926; 0.9174] 1.8
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.8
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.8
## Johnson, te Nijenhuis & Bouchard2008 0.7300 [ 0.4489; 0.8796] 1.8
## Johnson, te Nijenhuis & Bouchard2008 0.9300 [ 0.8376; 0.9707] 1.8
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [ 0.9758; 0.9959] 1.8
## Floyd et al.2010 0.9900 [ 0.9567; 0.9977] 0.7
## Kaufman et al.2012 0.8600 [ 0.7650; 0.9184] 4.5
## Salthouse2013 0.9100 [ 0.6399; 0.9800] 0.6
## Flanagan2000 0.8300 [ 0.5115; 0.9480] 0.9
## Floyd et al.2007 0.7770 [ 0.6472; 0.8630] 5.1
## Parkin & Beaujean2012 0.9100 [ 0.7984; 0.9612] 1.9
## Vanderwood et al.2002 0.9440 [ 0.9070; 0.9665] 5.3
## Wang et al.2021 0.5960 [ 0.1125; 0.8513] 1.1
## Snow et al.1977 0.9900 [ 0.9732; 0.9963] 1.5
## McGrew & Woodcock2001 A 0.9900 [ 0.9641; 0.9972] 0.9
## McGrew & Woodcock2001 B 0.9900 [ 0.9641; 0.9972] 0.9
## McGrew & Woodcock2001 C 0.9590 [ 0.8584; 0.9886] 0.9
## McGrew & Woodcock2001 D 0.9900 [ 0.9640; 0.9973] 0.9
## Woodcock1978 0.9790 [ 0.9290; 0.9939] 0.9
## Undheim1976 0.9900 [ 0.9636; 0.9973] 0.9
## Grigorenko et al.2004 0.6810 [ 0.2827; 0.8790] 1.2
## %W(random)
## DeVries & Kohlberg1974 1.1
## Devries 1974 A 1.1
## Devries1974 B 1.2
## Hathaway1972 A 1.1
## Hathaway1972 B 1.1
## Rindermann & Ackermann2020 A 1.0
## Rindermann & Ackermann2020 B 1.2
## Lim1988 1.5
## Stone1992 1.3
## Byrd & Buckhalt1991 1.1
## Tirre & Field2002 A 1.5
## Tirre & Field2002 B 1.5
## Tirre & Field2002 C 1.4
## Wothke et al.1991 1.5
## Williamson1969 1.4
## Kettner1976 A 1.5
## Kettner1976 B 1.5
## Palmer et al.1990 1.5
## Deary et al.1989 1.3
## Luo, Thompson & Detterman2003 1.6
## Carey1992 1.6
## Naglieri & Jensen1987 1.4
## Engelhardt2018 1.6
## Stauffer, Ree & Carretta1996 1.5
## Keith, Kranzler & Flanagan2001 1.4
## Deary et al.2004 1.2
## Acton & Schroeder2001 1.6
## Meyer et al.2010 1.6
## Johnson et al.2004 1.5
## Johnson et al.2004 1.5
## Johnson et al.2004 1.5
## Quiroga et al.2019 1.4
## Floyd et al.2013 1.4
## Floyd et al.2013 1.4
## Floyd et al.2013 1.4
## Floyd et al.2013 1.3
## Floyd et al.2013 1.2
## Peters, Kyngdon & Stillwell2021 1.3
## Reynolds et al.2015 1.4
## Valerius & Sparfeldt2014 A 1.6
## Valerius & Sparfeldt2014 B 1.6
## Valerius & Sparfeldt2014 C 1.6
## Quiroga et al.2015 1.4
## Swagerman et al.2016 1.5
## Johnson, te Nijenhuis & Bouchard2008 1.5
## Johnson, te Nijenhuis & Bouchard2008 1.5
## Johnson, te Nijenhuis & Bouchard2008 1.5
## Johnson, te Nijenhuis & Bouchard2008 1.5
## Johnson, te Nijenhuis & Bouchard2008 1.5
## Johnson, te Nijenhuis & Bouchard2008 1.5
## Johnson, te Nijenhuis & Bouchard2008 1.5
## Johnson, te Nijenhuis & Bouchard2008 1.5
## Johnson, te Nijenhuis & Bouchard2008 1.5
## Johnson, te Nijenhuis & Bouchard2008 1.5
## Floyd et al.2010 1.3
## Kaufman et al.2012 1.7
## Salthouse2013 1.3
## Flanagan2000 1.4
## Floyd et al.2007 1.7
## Parkin & Beaujean2012 1.6
## Vanderwood et al.2002 1.7
## Wang et al.2021 1.4
## Snow et al.1977 1.5
## McGrew & Woodcock2001 A 1.4
## McGrew & Woodcock2001 B 1.4
## McGrew & Woodcock2001 C 1.4
## McGrew & Woodcock2001 D 1.4
## Woodcock1978 1.4
## Undheim1976 1.4
## Grigorenko et al.2004 1.5
##
## Number of studies combined: k = 70
##
## COR 95%-CI z p-value
## Fixed effect model 0.9543 [0.9486; 0.9594] 60.92 0
## Random effects model 0.9587 [0.9430; 0.9701] 22.92 < 0.0001
##
## Quantifying heterogeneity:
## tau^2 = 0.4057 [0.2660; 0.6039]; tau = 0.6370 [0.5157; 0.7771];
## I^2 = 85.8% [82.7%; 88.3%]; H = 2.66 [2.41; 2.93]
##
## Test of heterogeneity:
## Q d.f. p-value
## 486.62 69 < 0.0001
##
## Details on meta-analytical method:
## - Inverse variance method
## - Sidik-Jonkman estimator for tau^2
## - Q-profile method for confidence interval of tau^2 and tau
## - Fisher's z transformation of correlationsDBZ <- metacor(rG,
sqrt(n),
data = JOGDB,
studlab = SY,
sm = "ZCOR",
method.tau = "SJ"); DBZ## COR 95%-CI %W(fixed)
## Hathaway1972 A 0.8920 [0.4668; 0.9822] 0.9
## Lim1988 0.9040 [0.7767; 0.9603] 3.7
## Byrd & Buckhalt1991 0.9900 [0.9273; 0.9987] 0.8
## Kettner1976 B 0.9900 [0.9718; 0.9965] 2.8
## Palmer et al.1990 0.9750 [0.9360; 0.9904] 3.4
## Luo, Thompson & Detterman2003 0.8700 [0.7141; 0.9437] 4.1
## Naglieri & Jensen1987 0.9900 [0.9661; 0.9971] 2.1
## Engelhardt2018 0.9800 [0.9569; 0.9908] 5.1
## Keith, Kranzler & Flanagan2001 0.9900 [0.9647; 0.9972] 1.9
## Deary et al.2004 0.9200 [0.6052; 0.9860] 1.0
## Johnson et al.2004 0.9900 [0.9749; 0.9960] 3.6
## Johnson et al.2004 0.9900 [0.9749; 0.9960] 3.6
## Quiroga et al.2019 0.8320 [0.4818; 0.9530] 1.7
## Floyd et al.2013 0.9700 [0.9061; 0.9906] 2.3
## Floyd et al.2013 0.9500 [0.8297; 0.9860] 1.9
## Floyd et al.2013 0.9900 [0.9625; 0.9974] 1.8
## Floyd et al.2013 0.9200 [0.6619; 0.9831] 1.2
## Valerius & Sparfeldt2014 B 0.9200 [0.8205; 0.9654] 4.2
## Johnson, te Nijenhuis & Bouchard2008 0.8900 [0.7515; 0.9534] 3.9
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [0.9758; 0.9959] 3.9
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [0.9758; 0.9959] 3.9
## Johnson, te Nijenhuis & Bouchard2008 0.9300 [0.8376; 0.9707] 3.9
## Johnson, te Nijenhuis & Bouchard2008 0.9900 [0.9758; 0.9959] 3.9
## Floyd et al.2010 0.9900 [0.9567; 0.9977] 1.4
## Kaufman et al.2012 0.8600 [0.7650; 0.9184] 9.6
## Floyd et al.2007 0.7770 [0.6472; 0.8630] 10.9
## Parkin & Beaujean2012 0.9100 [0.7984; 0.9612] 4.2
## Wang et al.2021 0.5960 [0.1125; 0.8513] 2.4
## McGrew & Woodcock2001 C 0.9590 [0.8584; 0.9886] 1.9
## Woodcock1978 0.9790 [0.9290; 0.9939] 2.0
## Undheim1976 0.9900 [0.9636; 0.9973] 1.8
## %W(random)
## Hathaway1972 A 2.4
## Lim1988 3.5
## Byrd & Buckhalt1991 2.2
## Kettner1976 B 3.3
## Palmer et al.1990 3.4
## Luo, Thompson & Detterman2003 3.5
## Naglieri & Jensen1987 3.1
## Engelhardt2018 3.6
## Keith, Kranzler & Flanagan2001 3.1
## Deary et al.2004 2.5
## Johnson et al.2004 3.5
## Johnson et al.2004 3.5
## Quiroga et al.2019 3.0
## Floyd et al.2013 3.2
## Floyd et al.2013 3.0
## Floyd et al.2013 3.0
## Floyd et al.2013 2.7
## Valerius & Sparfeldt2014 B 3.5
## Johnson, te Nijenhuis & Bouchard2008 3.5
## Johnson, te Nijenhuis & Bouchard2008 3.5
## Johnson, te Nijenhuis & Bouchard2008 3.5
## Johnson, te Nijenhuis & Bouchard2008 3.5
## Johnson, te Nijenhuis & Bouchard2008 3.5
## Floyd et al.2010 2.8
## Kaufman et al.2012 3.8
## Floyd et al.2007 3.9
## Parkin & Beaujean2012 3.5
## Wang et al.2021 3.2
## McGrew & Woodcock2001 C 3.0
## Woodcock1978 3.1
## Undheim1976 3.0
##
## Number of studies combined: k = 31
##
## COR 95%-CI z p-value
## Fixed effect model 0.9543 [0.9457; 0.9616] 41.67 0
## Random effects model 0.9638 [0.9444; 0.9766] 17.77 < 0.0001
##
## Quantifying heterogeneity:
## tau^2 = 0.3074 [0.1635; 0.5884]; tau = 0.5544 [0.4043; 0.7671];
## I^2 = 84.6% [79.1%; 88.6%]; H = 2.55 [2.19; 2.96]
##
## Test of heterogeneity:
## Q d.f. p-value
## 194.52 30 < 0.0001
##
## Details on meta-analytical method:
## - Inverse variance method
## - Sidik-Jonkman estimator for tau^2
## - Q-profile method for confidence interval of tau^2 and tau
## - Fisher's z transformation of correlationsKMZDATS <- escalc(measure = "ZCOR", ri = rG, ni = n, data = JOGKM)
DBZDATS <- escalc(measure = "ZCOR", ri = rG, ni = n, data = JOGDB)
KMZMAS <- rma.uni(yi = yi, vi = vi, data = KMZDATS, method = "ML"); KMZMAS; z2r(KMZMAS$b)##
## Random-Effects Model (k = 70; tau^2 estimator: ML)
##
## tau^2 (estimated amount of total heterogeneity): 0.4720 (SE = 0.0808)
## tau (square root of estimated tau^2 value): 0.6870
## I^2 (total heterogeneity / total variability): 99.49%
## H^2 (total variability / sampling variability): 196.61
##
## Test for Heterogeneity:
## Q(df = 69) = 12743.7594, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 1.9189 0.0826 23.2190 <.0001 1.7569 2.0809 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1## [,1]
## intrcpt 0.9578289DBZMAS <- rma.uni(yi = yi, vi = vi, data = DBZDATS, method = "ML"); DBZMAS; z2r(DBZMAS$b)##
## Random-Effects Model (k = 31; tau^2 estimator: ML)
##
## tau^2 (estimated amount of total heterogeneity): 0.3584 (SE = 0.0924)
## tau (square root of estimated tau^2 value): 0.5986
## I^2 (total heterogeneity / total variability): 99.37%
## H^2 (total variability / sampling variability): 158.16
##
## Test for Heterogeneity:
## Q(df = 30) = 5732.9565, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 2.0064 0.1084 18.5162 <.0001 1.7940 2.2188 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1## [,1]
## intrcpt 0.9644786Subgroup Analyses
Matching means that the tests are of the same general variety, i.e., achievement and achievement or elementary cognitive and elementary cognitive. If they do not match, then they are of a configuration like achievement and elementary cognitive or so on and so forth. The matching specifics break this down to a more granular level.
#Matching
ZORSUBM <- subgroup.analysis.mixed.effects(ZCOR, subgroups = JOG$Match)
forest(ZORSUBM)
#Matching Specifics
ZORSUBMT <- subgroup.analysis.mixed.effects(ZCOR, subgroups = JOG$MatchType)
forest(ZORSUBMT)
The psychometric sampling error tag was classified in a way that may not correspond very well to the actual concept. Psychometric sampling error is when the overrepresentation of certain ability factors or subtests impacts the identity of a factor. “High” (bad) and “Low” (good) classifications were based on the number of factors and the indicators, with a “Low” classification given if the number of indicators with an average of three indicators was three or greater or if the number of factors was four or greater without the average number of indicators needing to be three or more. If either test in the analysis failed to meet this requirement, psychometric sampling error was considered high. The “Quality” index includes the psychometric sampling error judgment in addition to a sample size requirement of at least n = 200; this is intended to proxy typical sampling error. Adjusting this to Gorsuch’s recommendation to have five times the number of manifest variables yields a practically identical result. It should be noted that this index of psychometric sampling error is not without faults; it should be obvious that a battery of tests can still show considerable psychometric sampling error even if the number of manifest variables and factors is high because they can invoke similar processes and have similar content and identities. This was likely the case for the studies by Carey, Abrahams et al., and Wolfe et al., since the ECAT factors were all very similar while the ASVAB factors were more diverse. As such, the note should be that my classification of psychometric sampling error here is an approximation that may not represent the real phenomenon sufficiently well, and likely does not, as many of these cases will show. It should also be noted that it is of course possible for a smaller battery to have less psychometric sampling error than a larger one. Notably, higher-order models were used when possible in order to keep psychometric sampling error low.
#Psychometric sampling error
ZORPSE <- subgroup.analysis.mixed.effects(ZCOR, subgroups = JOG$PSE)
forest(ZORPSE)
#Quality
ZORSUBQ <- subgroup.analysis.mixed.effects(ZCOR, subgroups = JOG$Quality)
forest(ZORSUBQ)
Unidimensionality was defined by the model yielding evidence for local independence, either by good fit or an actual inspection of residual correlations. Where possible, the latter was preferred. There was limited variation in this outcome, so this test is more perfunctory than anything, and the finding of multidimensionality does not mean that the battery was not principally unidimensional, it just means the model fit poorly, though that does not preclude potential modifications that would be acceptable, leave the battery unidimensional, and perhaps be more interpretable. Due to how JCFAs are performed, the fit with testing if there is ‘just one g’ is expected to be subpar because battery relationships - which are likely for group factors as well as g - were modeled practically only through g; extending this would not likely change the g relationships much and where this was tested, it did not. It sometimes increased and sometimes decreased it, but not meaningfully without overfitting of the sort that would make the model uninterpretable and likely unable to be generalized.
#Unidimensionality
ZORLI <- subgroup.analysis.mixed.effects(ZCOR, subgroups = JOG$LI)
forest(ZORLI)
Check the data file for the meanings of these labels, though it should be obvious what they are from the study names. These subgroup analyses should be taken with a pound of salt because shared authorship usually comes with other sources of heterogeneity, as in the case of Rindermann & Ackermann, whose data yielded high psychometric sampling error with very modest sample sizes or in Quiroga et al. (2015) whose estimate was lower than it should have been because they used sumscores to represent group factors.
#Shared Samples
ZORAU <- subgroup.analysis.mixed.effects(ZCOR, subgroups = JOG$SharedSample)
forest(ZORAU)
#Shared Authorship
ZORSA <- subgroup.analysis.mixed.effects(ZCOR, subgroups = JOG$SharedAuthor)
forest(ZORSA)
Meta-regression
#Age
ZCORRMAA <- rma.uni(yi = yi, vi = vi, data = RMADAT, method = "ML", mods = ~ Age); ZCORRMAA; z2r(ZCORRMAA$b)## Warning in rma.uni(yi = yi, vi = vi, data = RMADAT, method = "ML", mods = ~Age):
## Studies with NAs omitted from model fitting.##
## Mixed-Effects Model (k = 55; tau^2 estimator: ML)
##
## tau^2 (estimated amount of residual heterogeneity): 0.5540 (SE = 0.1068)
## tau (square root of estimated tau^2 value): 0.7443
## I^2 (residual heterogeneity / unaccounted variability): 99.69%
## H^2 (unaccounted variability / sampling variability): 323.93
## R^2 (amount of heterogeneity accounted for): 10.27%
##
## Test for Residual Heterogeneity:
## QE(df = 53) = 15562.1599, p-val < .0001
##
## Test of Moderators (coefficient 2):
## QM(df = 1) = 6.2403, p-val = 0.0125
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## intrcpt 1.4444 0.1931 7.4787 <.0001 1.0658 1.8229 ***
## Age 0.0234 0.0094 2.4981 0.0125 0.0050 0.0417 *
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1## [,1]
## intrcpt 0.89457337
## Age 0.02338284Conclusions
It appears there is only one general factor, g, and it is indexed much the same when tests do not share content. This suggests that, more than the positive manifold, the indifference of the indicator is a stylized fact for intelligence research.
The relationship between g factors may be moderated by age, but the effect was insignificant with p-scaling and only modest otherwise. One standard deviation of age in this meta-analysis was 10.89 years (min-max, 5.22-53.60, \(\mu\) = 17.55, K = 55); this outcome could be more significant with a greater range, more homogeneity in the tests and samples used, and a larger K. It is very likely that heterogeneity negatively affected the significance of the end result in no small part because of range restriction where correlations greater than one are impossible.
References
Wong, C.-M. T., Day, J. D., Maxwell, S. E., & Meara, N. M. (1995). A multitrait-multimethod study of academic and social intelligence in college students. Journal of Educational Psychology, 87(1), 117–133. https://doi.org/10.1037/0022-0663.87.1.117
Buczyłowska, D., Petermann, F., & Daseking, M. (2020). Executive functions and intelligence from the CHC theory perspective: Investigating the correspondence between the WAIS-IV and the NAB Executive Functions Module. Journal of Clinical and Experimental Neuropsychology, 42(3), 240–250. https://doi.org/10.1080/13803395.2019.1705250