Reliability Analysis
Asghar Minaei
10/1/2022
- Your Coefficient Alpha Is Probably Wrong,but Which Coefficient Omega Is Right? A Tutorial on Using R to Obtain Better Reliability Estimates
- Daniel McNeish(2017). Thanks Coefficient Alpha, We’ll Take It From Here
- Eunseong Cho and Seonghoon Kim(2020).Cronbach’s Coefficient Alpha: Well Known but Poorly Understood
Your Coefficient Alpha Is Probably Wrong,but Which Coefficient Omega Is Right? A Tutorial on Using R to Obtain Better Reliability Estimates
What Is Reliability?
Observed scores on any given psychological test or scale are determined by a combination of systematic (signal) and random (noise) influences. Reliability is defined as a population-based quantification of measurement precision (e.g., Mellenbergh, 1996) as a function of the signal-to-noise ratio. Measurement error, or unreliability, produces biased estimates of effects meant to represent true associations among constructs (Bollen, 1989; Cole & Preacher, 2014), and measurement error is a culprit in the replication crisis (Loken & Gelman, 2017). Thus, using tests with maximally reliable scores and using statistical methods to account for measurement error (e.g., Savalei, 2019) can help psychology progress as a replicable science.
It is important to recognize that the CTT true scoredoes not necessarily equate to a construct score (Borsboom, 2005). Thus, a true score may be determined by a construct that a test is designed to measure (the target construct) as well as by other systematic influences. Most often, researchers want to know how reliably a test measures the target construct itself, and for this reason it is important to establish the dimensionality, or internal structure, of a test before estimating reliability (Savalei & Reise, 2019). poor fit of a onefactor model is evidence of multidimensionality.
If reliability is estimated using the parameters of an incorrect (i.e., misspecified) factor model, then the reliability estimate is likely to be biased with respect to the measurement of the target construct.
In the preliminary stages of scale development, EFA is valuable for uncovering systematic influences on item responses that might map onto hypothesized constructs, whereas specification of a CFA model implies that the item-construct relations underpinning certain reliability estimates have been more well established (Flora & Flake, 2017; Zinbarg, Yovel, Revelle, & McDonald, 2006).
After addressing the unidimensional case, I present omega estimates of total-score reliability for multidimensional tests; the online supplement addresses reliability assessment for subscale scores.
Reliability of Unidimensional Tests:Omega to Alpha
Coefficient omega
The reliability of the total score on a unidimensional test can be estimated from parameter estimates of a one-factor model fitted to the item scores. I refer to this reliability estimate as ωu.
This formula gives a form of coefficient omega that is appropriate under the strong assumption that the one-factor model is correct,that is, the set of items is unidimensional, so that ωu represents the proportion of total-score variance that is due to the single factor.
The \(\hat{\sigma}^2\) term in the denominator of ωu can be calculated either as the sample variance of the total score X or as the model-implied variance of X. Generally, this choice is unlikely to have a large effect on the magnitude of omega estimates if the estimated model is not badly misspecified.
alpha is an estimate of total-score reliability for the measurement of a single construct common to all items in a test under the conditions that (a) a one-factor model is correct (i.e., the test is unidimensional), (b) the factor loadings are equal across all items (i.e., essential tau equivalence), and (c) the errors ej are independent across items.
Overall, estimates of ωu are unbiased with varying factor loadings (i.e., violation of tau equivalence), when alpha underestimates population reliability (e.g., Zinbarg, Revelle, Yovel, & Li, 2005). Furthermore, Yang and Green (2010) showed that ωu estimates are largely robust when the estimated model contains more factors than the true model, even with samples as small as 50.
Example calculation of ωu in R
To demonstrate calculation of ωu using R, I use data for five items measuring the personality trait openness as completed by 2,800 participants in the Synthetic Aperture Personality Assessment project (Revelle, Wilt, & Rosenthal, 2010). These data are available on this Tutorial’s OSF site as well as in the psych package for R (Revelle, 2020).These items have a 6-point, ordered categorical Likert-type response scale and thus are treated as continuous variables for CFA and reliability estimation.
library(psych)
data(bfi)
# load data file
bfi <- bfi
names(bfi)## [1] "A1" "A2" "A3" "A4" "A5" "C1"
## [7] "C2" "C3" "C4" "C5" "E1" "E2"
## [13] "E3" "E4" "E5" "N1" "N2" "N3"
## [19] "N4" "N5" "O1" "O2" "O3" "O4"
## [25] "O5" "gender" "education" "age"# Select items of Openness subscale.
ind <- grep("O", colnames(bfi))
oppeness <- na.omit(bfi[,ind])
head(oppeness)## O1 O2 O3 O4 O5
## 61617 3 6 3 4 3
## 61618 4 2 4 3 3
## 61620 4 2 5 5 2
## 61621 3 3 4 3 5
## 61622 3 3 4 3 3
## 61623 4 3 5 6 1# Reversing item O2 and O5
q <- c("O2","O5")
reverse <- function(x){
x.reversed <- 7 +0 - x
}
oppeness[, c("O2R", "O5R")] <- reverse(oppeness[,c("O2","O5")])
# compare original and reversed responses
oppeness[1:6,]## O1 O2 O3 O4 O5 O2R O5R
## 61617 3 6 3 4 3 1 4
## 61618 4 2 4 3 3 5 4
## 61620 4 2 5 5 2 5 5
## 61621 3 3 4 3 5 4 2
## 61622 3 3 4 3 3 4 4
## 61623 4 3 5 6 1 4 6# compute alpha coefficient
alpha(oppeness[, -c(2,5)])##
## Reliability analysis
## Call: alpha(x = oppeness[, -c(2, 5)])
##
## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd median_r
## 0.6 0.61 0.57 0.24 1.6 0.012 4.6 0.81 0.23
##
## lower alpha upper 95% confidence boundaries
## 0.58 0.6 0.63
##
## Reliability if an item is dropped:
## raw_alpha std.alpha G6(smc) average_r S/N alpha se var.r med.r
## O1 0.54 0.54 0.48 0.23 1.2 0.014 0.0087 0.23
## O3 0.50 0.50 0.44 0.20 1.0 0.015 0.0065 0.20
## O4 0.61 0.62 0.56 0.29 1.7 0.012 0.0040 0.29
## O2R 0.57 0.57 0.51 0.25 1.3 0.014 0.0077 0.21
## O5R 0.52 0.53 0.48 0.22 1.1 0.015 0.0109 0.21
##
## Item statistics
## n raw.r std.r r.cor r.drop mean sd
## O1 2726 0.61 0.65 0.51 0.39 4.8 1.1
## O3 2726 0.68 0.69 0.59 0.45 4.4 1.2
## O4 2726 0.50 0.52 0.29 0.22 4.9 1.2
## O2R 2726 0.66 0.60 0.44 0.34 4.3 1.6
## O5R 2726 0.67 0.66 0.52 0.42 4.5 1.3
##
## Non missing response frequency for each item
## 1 2 3 4 5 6 miss
## O1 0.01 0.04 0.08 0.22 0.33 0.33 0
## O3 0.03 0.05 0.10 0.28 0.34 0.20 0
## O4 0.02 0.05 0.06 0.17 0.32 0.39 0
## O2R 0.06 0.10 0.15 0.14 0.26 0.29 0
## O5R 0.02 0.07 0.13 0.19 0.32 0.27 0# load library lavaan
library(lavaan)
# Specify model
mod1f <- "
openness =~ O1 + O2R + O3 + O4 + O5R
"
# Estimate model
fit.one.f <- cfa(mod1f, data=oppeness, std.lv=T, missing='direct',
estimator='MLR')
# The results can be viewed using the summary
summary(fit.one.f, fit.measures=T, standardized=T)## lavaan 0.6-9 ended normally after 22 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 15
##
## Number of observations 2726
## Number of missing patterns 1
##
## Model Test User Model:
## Standard Robust
## Test Statistic 80.583 67.248
## Degrees of freedom 5 5
## P-value (Chi-square) 0.000 0.000
## Scaling correction factor 1.198
## Yuan-Bentler correction (Mplus variant)
##
## Model Test Baseline Model:
##
## Test statistic 1375.389 1049.226
## Degrees of freedom 10 10
## P-value 0.000 0.000
## Scaling correction factor 1.311
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.945 0.940
## Tucker-Lewis Index (TLI) 0.889 0.880
##
## Robust Comparative Fit Index (CFI) 0.945
## Robust Tucker-Lewis Index (TLI) 0.890
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -22078.661 -22078.661
## Scaling correction factor 1.167
## for the MLR correction
## Loglikelihood unrestricted model (H1) -22038.369 -22038.369
## Scaling correction factor 1.174
## for the MLR correction
##
## Akaike (AIC) 44187.322 44187.322
## Bayesian (BIC) 44275.980 44275.980
## Sample-size adjusted Bayesian (BIC) 44228.321 44228.321
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.074 0.068
## 90 Percent confidence interval - lower 0.061 0.055
## 90 Percent confidence interval - upper 0.089 0.081
## P-value RMSEA <= 0.05 0.002 0.012
##
## Robust RMSEA 0.074
## 90 Percent confidence interval - lower 0.059
## 90 Percent confidence interval - upper 0.090
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.029 0.029
##
## Parameter Estimates:
##
## Standard errors Sandwich
## Information bread Observed
## Observed information based on Hessian
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## openness =~
## O1 0.616 0.029 21.346 0.000 0.616 0.546
## O2R 0.701 0.041 17.170 0.000 0.701 0.449
## O3 0.792 0.032 24.649 0.000 0.792 0.649
## O4 0.357 0.030 11.749 0.000 0.357 0.294
## O5R 0.685 0.036 19.194 0.000 0.685 0.517
##
## Intercepts:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .O1 4.819 0.022 223.110 0.000 4.819 4.273
## .O2R 4.300 0.030 143.782 0.000 4.300 2.754
## .O3 4.439 0.023 189.916 0.000 4.439 3.637
## .O4 4.898 0.023 210.231 0.000 4.898 4.027
## .O5R 4.516 0.025 177.976 0.000 4.516 3.409
## openness 0.000 0.000 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .O1 0.893 0.037 23.923 0.000 0.893 0.702
## .O2R 1.947 0.068 28.607 0.000 1.947 0.798
## .O3 0.862 0.050 17.172 0.000 0.862 0.579
## .O4 1.352 0.052 25.967 0.000 1.352 0.914
## .O5R 1.286 0.059 21.822 0.000 1.286 0.732
## openness 1.000 1.000 1.000residuals(fit.one.f, type='cor')## $type
## [1] "cor.bollen"
##
## $cov
## O1 O2R O3 O4 O5R
## O1 0.000
## O2R -0.026 0.000
## O3 0.037 -0.024 0.000
## O4 0.013 -0.052 0.000 0.000
## O5R -0.044 0.090 -0.022 0.027 0.000
##
## $mean
## O1 O2R O3 O4 O5R
## 0 0 0 0 0The fit statistics reported under the Robust column in the summary indicate that this one-factor model has a marginal fit to the data; although the CFI of .94 suggests adequate fit, the TLI of .88 is lower than desired, and the RMSEA is somewhat high at 0.08.
These estimates of \(\hat{\lambda}\) vary substantially (from .36 to.79), suggesting that tau equivalence is not tenable for this openness test and thus that ωu is a more appropriate reliability estimate than alpha.6 The ωu estimate can be obtained by executing the reliability function from the semTools package (i.e., semTools::reliability) on the fit1f one-factor model object:
library(semTools)
reliability(fit.one.f)## openness
## alpha 0.6025464
## omega 0.6103741
## omega2 0.6103741
## omega3 0.6098642
## avevar 0.2484010The results listed in the omega and omega2 rows give the ωu estimate in which the denominator equals the model-implied variance of the total score X, and the results listed in the omega3 row are based on a variation in which the denominator equals the observed, sample variance of X, as described earlier.
The omega3 estimate is referred to as “hierarchical omega” in the help documentation for semTools::reliability (Jorgensen et al., 2020) and in Kelley and Pornprasertmanit (2016), whereas this Tutorial follows most of the psychometric literature (e.g., Rodriguez et al., 2016; Zinbarg et al., 2006) by reserving the term omega-hierarchical for estimates based on a bifactor model, which is presented later in this Tutorial.
Kelley and Pornprasertmanit (2016) reviewed different approaches to constructing CIs around omega estimates, ultimately recommending bootstrapping.
Estimate confidence interval
library(MBESS)
#ci.reliability(data=oppeness[, c(1,3,4,6,7)], type="omega", interval.type="perc")In this case, the fit.one.f model has a small but notable residual correlation (r = .10) between the O2 and O5 items, which is not surprising because these were the two reverse-coded items. Thus, it may be important to account for the corresponding error covariance in the calculation of ωu. To include this term as a free parameter, the one-factor model can be respecified as a new model, called mod1fR, as follows:
# Respecified model
mod1fR <- "
openness =~ O1 + O2R + O3 + O4 + O5R
O2R ~~ O5R
"
# Estimate respecified model
fit.one.fR <- cfa(mod1fR, data=oppeness, std.lv=T, missing='direct',
estimator='MLR')
# The results can be viewed using the summary
summary(fit.one.fR, fit.measures=T, standardized=T)## lavaan 0.6-9 ended normally after 25 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 16
##
## Number of observations 2726
## Number of missing patterns 1
##
## Model Test User Model:
## Standard Robust
## Test Statistic 17.079 14.121
## Degrees of freedom 4 4
## P-value (Chi-square) 0.002 0.007
## Scaling correction factor 1.209
## Yuan-Bentler correction (Mplus variant)
##
## Model Test Baseline Model:
##
## Test statistic 1375.389 1049.226
## Degrees of freedom 10 10
## P-value 0.000 0.000
## Scaling correction factor 1.311
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.990 0.990
## Tucker-Lewis Index (TLI) 0.976 0.976
##
## Robust Comparative Fit Index (CFI) 0.991
## Robust Tucker-Lewis Index (TLI) 0.978
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -22046.908 -22046.908
## Scaling correction factor 1.166
## for the MLR correction
## Loglikelihood unrestricted model (H1) -22038.369 -22038.369
## Scaling correction factor 1.174
## for the MLR correction
##
## Akaike (AIC) 44125.817 44125.817
## Bayesian (BIC) 44220.386 44220.386
## Sample-size adjusted Bayesian (BIC) 44169.549 44169.549
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.035 0.030
## 90 Percent confidence interval - lower 0.019 0.016
## 90 Percent confidence interval - upper 0.052 0.047
## P-value RMSEA <= 0.05 0.921 0.977
##
## Robust RMSEA 0.034
## 90 Percent confidence interval - lower 0.016
## 90 Percent confidence interval - upper 0.053
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.014 0.014
##
## Parameter Estimates:
##
## Standard errors Sandwich
## Information bread Observed
## Observed information based on Hessian
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## openness =~
## O1 0.634 0.029 21.969 0.000 0.634 0.562
## O2R 0.594 0.040 14.768 0.000 0.594 0.380
## O3 0.843 0.033 25.259 0.000 0.843 0.691
## O4 0.359 0.031 11.575 0.000 0.359 0.295
## O5R 0.602 0.034 17.669 0.000 0.602 0.454
##
## Covariances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .O2R ~~
## .O5R 0.310 0.046 6.715 0.000 0.310 0.182
##
## Intercepts:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .O1 4.819 0.022 223.110 0.000 4.819 4.273
## .O2R 4.300 0.030 143.782 0.000 4.300 2.754
## .O3 4.439 0.023 189.916 0.000 4.439 3.637
## .O4 4.898 0.023 210.231 0.000 4.898 4.027
## .O5R 4.516 0.025 177.976 0.000 4.516 3.409
## openness 0.000 0.000 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .O1 0.870 0.038 22.838 0.000 0.870 0.684
## .O2R 2.085 0.064 32.701 0.000 2.085 0.855
## .O3 0.778 0.053 14.560 0.000 0.778 0.523
## .O4 1.351 0.052 25.828 0.000 1.351 0.913
## .O5R 1.393 0.055 25.220 0.000 1.393 0.794
## openness 1.000 1.000 1.000residuals(fit.one.fR, type='cor')## $type
## [1] "cor.bollen"
##
## $cov
## O1 O2R O3 O4 O5R
## O1 0.000
## O2R 0.006 0.000
## O3 0.003 0.005 0.000
## O4 0.007 -0.033 -0.013 0.000
## O5R -0.017 0.000 0.000 0.045 0.000
##
## $mean
## O1 O2R O3 O4 O5R
## 0 0 0 0 0# Estimate Omega for respecified model
reliability(fit.one.fR)## openness
## alpha 0.6025464
## omega 0.5642414
## omega2 0.5642414
## omega3 0.5643914
## avevar 0.2319484Because the overall model fit was improved and the error covariance is conceptually meaningful (because reverse-scored items often share excess covariance), the revised ωu of .56 is more trustworthy than the first ωu as an estimate of the composite reliability of the openness scale with respect to the measurement of a single openness factor.
Unfortunately, the ci.reliability function cannot explicitly account for the error covariance to obtain a CI around this ωu estimate.
###Factor Analysis of Ordered, Categorical Items
Most often, item responses are scored with an ordered, categorical scale (e.g., Likert-type items scored with discrete integers) or a binary response scale (e.g., 1 = yes, 0 = no). These scales produce categorical data for which the traditional, linear factor-analytic model is technically incorrect; thus, fitting CFA models to the observed Pearson product-moment covariance (or correlation) matrix among item scores can produce inaccurate results (see Flora, LaBrish, & Chalmers, 2012).
One can fit a CFA model to polychoric correlations, rather than product-moment covariance or correlations, to account for items’ categorical nature.
Categorical omega
Because the factor loadings estimated from polychoric correlations represent the associations between the factor and latent-response variables rather than the observed item responses themselves, applying the ωu formula presented earlier in this context would give the proportion of variance explained by the factor relative to the variance of the sum of the latent-response variables instead of the total variance of the sum of the observed item responses; that is, ωu would represent the reliability of a hypothetical, latent total score instead of the reliability of the actual observed total score. To remedy this issue, Green and Yang (2009b) developed an alternative reliability estimate that is rescaled into the observed total score metric. This reliability estimate for unidimensional categorical items is referred to as ωu-cat.
Yang and Green established that, compared with ωu, ωu-cat produces more accurate reliability estimates for total score X with ordinal item-level data, especially when the univariate item-response distributions differ across items.
Because the psychoticism items have a 4-point response scale, I fitted the one-factor model to the interitem polychoric correlations using WLSMV, a robust weighted least squares estimator that is recommended over the ML estimator for CFA with polychoric correlations (Finney & DiStefano, 2013).
# read in data
library(rio)
potic <- import("potic.csv")
# Specify model
mod.one.fCat <- 'psyctcsm =~ DDP1 + DDP2 +
DDP3 + DDP4'
# Estimate model
fit.one.fCat <- cfa(mod.one.fCat, data=potic, std.lv=TRUE, ordered=T, estimator='WLSMV')
# Retrieving the results
summary(fit.one.fCat, fit.measures=TRUE, standardized=TRUE)## lavaan 0.6-9 ended normally after 10 iterations
##
## Estimator DWLS
## Optimization method NLMINB
## Number of model parameters 20
##
## Used Total
## Number of observations 498 500
##
## Model Test User Model:
## Standard Robust
## Test Statistic 7.117 14.910
## Degrees of freedom 2 2
## P-value (Chi-square) 0.028 0.001
## Scaling correction factor 0.480
## Shift parameter 0.089
## simple second-order correction
##
## Model Test Baseline Model:
##
## Test statistic 1635.647 1316.086
## Degrees of freedom 6 6
## P-value 0.000 0.000
## Scaling correction factor 1.244
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.997 0.990
## Tucker-Lewis Index (TLI) 0.991 0.970
##
## Robust Comparative Fit Index (CFI) NA
## Robust Tucker-Lewis Index (TLI) NA
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.072 0.114
## 90 Percent confidence interval - lower 0.020 0.065
## 90 Percent confidence interval - upper 0.132 0.171
## P-value RMSEA <= 0.05 0.200 0.019
##
## Robust RMSEA NA
## 90 Percent confidence interval - lower NA
## 90 Percent confidence interval - upper NA
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.033 0.033
##
## Parameter Estimates:
##
## Standard errors Robust.sem
## Information Expected
## Information saturated (h1) model Unstructured
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## psyctcsm =~
## DDP1 0.894 0.028 31.538 0.000 0.894 0.894
## DDP2 0.753 0.028 26.711 0.000 0.753 0.753
## DDP3 0.698 0.030 23.314 0.000 0.698 0.698
## DDP4 0.513 0.040 12.738 0.000 0.513 0.513
##
## Intercepts:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .DDP1 0.000 0.000 0.000
## .DDP2 0.000 0.000 0.000
## .DDP3 0.000 0.000 0.000
## .DDP4 0.000 0.000 0.000
## psyctcsm 0.000 0.000 0.000
##
## Thresholds:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## DDP1|t1 -0.716 0.062 -11.591 0.000 -0.716 -0.716
## DDP1|t2 -0.040 0.056 -0.716 0.474 -0.040 -0.040
## DDP1|t3 0.296 0.057 5.185 0.000 0.296 0.296
## DDP1|t4 1.034 0.069 15.065 0.000 1.034 1.034
## DDP2|t1 -0.580 0.060 -9.693 0.000 -0.580 -0.580
## DDP2|t2 0.208 0.057 3.668 0.000 0.208 0.208
## DDP2|t3 0.562 0.060 9.431 0.000 0.562 0.562
## DDP2|t4 1.096 0.070 15.570 0.000 1.096 1.096
## DDP3|t1 -1.203 0.074 -16.298 0.000 -1.203 -1.203
## DDP3|t2 -0.425 0.058 -7.318 0.000 -0.425 -0.425
## DDP3|t3 0.081 0.056 1.432 0.152 0.081 0.081
## DDP3|t4 0.913 0.066 13.909 0.000 0.913 0.913
## DDP4|t1 -1.605 0.092 -17.382 0.000 -1.605 -1.605
## DDP4|t2 -1.060 0.069 -15.285 0.000 -1.060 -1.060
## DDP4|t3 -0.521 0.059 -8.817 0.000 -0.521 -0.521
## DDP4|t4 0.431 0.058 7.406 0.000 0.431 0.431
##
## Variances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .DDP1 0.201 0.201 0.201
## .DDP2 0.433 0.433 0.433
## .DDP3 0.513 0.513 0.513
## .DDP4 0.737 0.737 0.737
## psyctcsm 1.000 1.000 1.000
##
## Scales y*:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## DDP1 1.000 1.000 1.000
## DDP2 1.000 1.000 1.000
## DDP3 1.000 1.000 1.000
## DDP4 1.000 1.000 1.000# Estimate reliability
library(semTools)
reliability(fit.one.fCat)## psyctcsm
## alpha 0.7680870
## alpha.ord 0.8007496
## omega 0.7902953
## omega2 0.7902953
## omega3 0.7932682
## avevar 0.5289638Estimate of Cronbach’s alpha (.80) that differs from the estimate reported by semTools for the psychoticism scale (i.e., .77); this alpha is an example of ordinal alpha (Zumbo, Gadermann, & Zeisser, 2007) because it is based on the model estimated using polychoric correlations, whereas the first alpha estimate used the traditional calculation from interitem productmoment covariances. Note that ordinal alpha is a reliability estimate for the sum of the continuous, latent-response variables rather than for X, the sum of the observed, categorical item-response variables. Additionally, ordinal alpha still carries the assumption of equal factor loadings (i.e., tau equivalence). For these reasons, I advocate ignoring the alpha results reported by semTools::reliability when the factor model has been fitted using polychoric correlations.
Results listed in the omega and omega2 rows give the ωu-cat estimate based on Green and Yang’s (2009b) formula, in which the denominator equals the model implied variance of the total score X, and results listed in the omega3 row are based on a variation of Green and Yang’s formula in which the denominator equals the observed, sample variance of X. Thus, the ωu-cat estimate indicates that .79 of the scale’s total-score variance is due to the single psychoticism factor.
The ci.reliability function can also be used to obtain a confidence interval for ωu-cat.
#ci.reliability(data=potic, type="categorical",
# interval.type="perc", B=100)Often, tests are designed to measure a single construct but end up having a multidimensional structure, especially as the content of the test broadens. Occasionally, multidimensionality is intentional, as when a test is designed to produce subscale scores in addition to a total score. In other situations, the breadth of the construct’s definition or the format of items produces unintended multidimensionality, even if a general target construct that influences all items is still present. In either case, the one-factor model presented earlier is incorrect, and thus it is generally inappropriate to use alpha or ωu to represent the reliability of a total score from a multidimensional test. Instead, reliability estimates for observed scores derived from multidimensional tests should be interpretable with respect to the target constructs.
Bifactor models
One way to represent such a multidimensional structure is with a bifactor model, in which a general factor influences all items and specific factors (also known as group factors) capture covariation among subsets of items that remains over and above the covariance due to the general factor.
The general factor must be uncorrelated with the specific factors to guarantee model identification (Yung, Thissen, & McLeod, 1999), whereas in other CFA models, all factors freely correlate with each other (allowing the general factor in a bifactor model to correlate with one of the specific factors causes errors such as nonconvergence or improper solutions).
Omega-hierarchical
When item-response data are well represented by a bifactor model, a reliability measure known as omega hierarchical (or ωh) represents the proportion of totalscore variance due to a single, general construct that influences all items, despite the multidimensional nature of the item set (Rodriguez, Reise, & Haviland, 2016). the Numerator of ωh is a function of the general-factor loadings only; the denominator again represents the estimated variance of the total score X. Therefore, ωh represents the extent to which the total score provides a reliable measure of a construct represented by a general factor that influences all items in a multidimensional scale over and above extraneous influences captured by the specific factors. Zinbarg et al., 2005).
Example calculation of ωh in R
To demonstrate the estimation of ωh using R, I use data that Flake, Ferland, & Flora, 2017 collected by administering the PCS to 154 students in an introductory statistics course.
# read in data
pcs <- import("pcs.csv")
names(pcs)## [1] "TE1" "TE2" "TE3" "TE4" "TE5" "OE1" "OE2" "OE3" "OE4" "LVA1"
## [11] "LVA2" "LVA3" "LVA4" "EM1" "EM2" "EM3" "EM4" "EM5" "EM6"modBf <- "
gen =~ TE1+TE2+TE3+TE4+TE
5+OE1+OE2+OE3+OE4+LVA1+LVA2+LVA3+
LVA4 +EM1+EM2+EM3+EM4+EM5+EM6
s1 =~ TE1 + TE2 + TE3 + TE4 + TE5
s2 =~ OE1 + OE2 + OE3 + OE4
s3 =~ LVA1 + LVA2 + LVA3 + LVA4
s4 =~ EM1 + EM2 + EM3 + EM4 + EM5 + EM6
"
#Specify model
fitBf <- cfa(modBf, data=pcs, std.lv=T, estimator='MLR', orthogonal=T)
#Retrieving the results
summary(fitBf, fit.measures=TRUE, standardized=T)## lavaan 0.6-9 ended normally after 36 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 57
##
## Used Total
## Number of observations 154 172
##
## Model Test User Model:
## Standard Robust
## Test Statistic 211.382 182.509
## Degrees of freedom 133 133
## P-value (Chi-square) 0.000 0.003
## Scaling correction factor 1.158
## Yuan-Bentler correction (Mplus variant)
##
## Model Test Baseline Model:
##
## Test statistic 2799.877 2260.239
## Degrees of freedom 171 171
## P-value 0.000 0.000
## Scaling correction factor 1.239
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.970 0.976
## Tucker-Lewis Index (TLI) 0.962 0.970
##
## Robust Comparative Fit Index (CFI) 0.978
## Robust Tucker-Lewis Index (TLI) 0.972
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -3456.819 -3456.819
## Scaling correction factor 1.269
## for the MLR correction
## Loglikelihood unrestricted model (H1) -3351.128 -3351.128
## Scaling correction factor 1.191
## for the MLR correction
##
## Akaike (AIC) 7027.638 7027.638
## Bayesian (BIC) 7200.744 7200.744
## Sample-size adjusted Bayesian (BIC) 7020.331 7020.331
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.062 0.049
## 90 Percent confidence interval - lower 0.046 0.031
## 90 Percent confidence interval - upper 0.077 0.065
## P-value RMSEA <= 0.05 0.108 0.520
##
## Robust RMSEA 0.053
## 90 Percent confidence interval - lower 0.032
## 90 Percent confidence interval - upper 0.071
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.038 0.038
##
## Parameter Estimates:
##
## Standard errors Sandwich
## Information bread Observed
## Observed information based on Hessian
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## gen =~
## TE1 1.041 0.071 14.715 0.000 1.041 0.843
## TE2 1.045 0.084 12.430 0.000 1.045 0.794
## TE3 0.848 0.090 9.392 0.000 0.848 0.721
## TE4 0.984 0.075 13.097 0.000 0.984 0.834
## TE5 1.004 0.082 12.264 0.000 1.004 0.819
## OE1 0.787 0.083 9.537 0.000 0.787 0.631
## OE2 0.819 0.080 10.207 0.000 0.819 0.695
## OE3 0.738 0.089 8.319 0.000 0.738 0.605
## OE4 0.742 0.087 8.512 0.000 0.742 0.636
## LVA1 0.937 0.084 11.180 0.000 0.937 0.796
## LVA2 0.863 0.071 12.205 0.000 0.863 0.797
## LVA3 0.816 0.085 9.649 0.000 0.816 0.719
## LVA4 0.865 0.098 8.802 0.000 0.865 0.695
## EM1 0.968 0.095 10.215 0.000 0.968 0.715
## EM2 0.930 0.078 11.957 0.000 0.930 0.800
## EM3 0.959 0.091 10.542 0.000 0.959 0.731
## EM4 0.885 0.091 9.679 0.000 0.885 0.732
## EM5 1.043 0.086 12.121 0.000 1.043 0.816
## EM6 1.108 0.100 11.054 0.000 1.108 0.750
## s1 =~
## TE1 0.351 0.114 3.071 0.002 0.351 0.285
## TE2 0.451 0.144 3.142 0.002 0.451 0.343
## TE3 0.402 0.170 2.360 0.018 0.402 0.342
## TE4 0.162 0.111 1.457 0.145 0.162 0.137
## TE5 0.269 0.154 1.747 0.081 0.269 0.219
## s2 =~
## OE1 0.626 0.107 5.860 0.000 0.626 0.502
## OE2 0.516 0.096 5.399 0.000 0.516 0.438
## OE3 0.673 0.107 6.291 0.000 0.673 0.552
## OE4 0.739 0.085 8.738 0.000 0.739 0.634
## s3 =~
## LVA1 0.253 0.107 2.357 0.018 0.253 0.215
## LVA2 0.573 0.081 7.081 0.000 0.573 0.529
## LVA3 0.422 0.104 4.051 0.000 0.422 0.372
## LVA4 0.528 0.104 5.074 0.000 0.528 0.424
## s4 =~
## EM1 0.506 0.152 3.324 0.001 0.506 0.374
## EM2 0.346 0.086 4.026 0.000 0.346 0.298
## EM3 0.567 0.121 4.682 0.000 0.567 0.432
## EM4 0.562 0.098 5.707 0.000 0.562 0.464
## EM5 0.479 0.097 4.930 0.000 0.479 0.375
## EM6 0.651 0.148 4.403 0.000 0.651 0.440
##
## Covariances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## gen ~~
## s1 0.000 0.000 0.000
## s2 0.000 0.000 0.000
## s3 0.000 0.000 0.000
## s4 0.000 0.000 0.000
## s1 ~~
## s2 0.000 0.000 0.000
## s3 0.000 0.000 0.000
## s4 0.000 0.000 0.000
## s2 ~~
## s3 0.000 0.000 0.000
## s4 0.000 0.000 0.000
## s3 ~~
## s4 0.000 0.000 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .TE1 0.318 0.067 4.777 0.000 0.318 0.209
## .TE2 0.434 0.088 4.913 0.000 0.434 0.251
## .TE3 0.501 0.104 4.827 0.000 0.501 0.363
## .TE4 0.397 0.066 5.987 0.000 0.397 0.285
## .TE5 0.421 0.073 5.799 0.000 0.421 0.281
## .OE1 0.546 0.106 5.129 0.000 0.546 0.350
## .OE2 0.451 0.075 6.003 0.000 0.451 0.325
## .OE3 0.490 0.117 4.183 0.000 0.490 0.330
## .OE4 0.263 0.071 3.718 0.000 0.263 0.194
## .LVA1 0.442 0.064 6.873 0.000 0.442 0.320
## .LVA2 0.100 0.062 1.607 0.108 0.100 0.085
## .LVA3 0.443 0.075 5.892 0.000 0.443 0.344
## .LVA4 0.523 0.088 5.954 0.000 0.523 0.337
## .EM1 0.639 0.100 6.388 0.000 0.639 0.349
## .EM2 0.365 0.060 6.049 0.000 0.365 0.271
## .EM3 0.482 0.090 5.383 0.000 0.482 0.280
## .EM4 0.364 0.083 4.395 0.000 0.364 0.249
## .EM5 0.318 0.063 5.032 0.000 0.318 0.195
## .EM6 0.534 0.117 4.564 0.000 0.534 0.244
## gen 1.000 1.000 1.000
## s1 1.000 1.000 1.000
## s2 1.000 1.000 1.000
## s3 1.000 1.000 1.000
## s4 1.000 1.000 1.000fitmeasures(fitBf)## npar fmin
## 57.000 0.686
## chisq df
## 211.382 133.000
## pvalue chisq.scaled
## 0.000 182.509
## df.scaled pvalue.scaled
## 133.000 0.003
## chisq.scaling.factor baseline.chisq
## 1.158 2799.877
## baseline.df baseline.pvalue
## 171.000 0.000
## baseline.chisq.scaled baseline.df.scaled
## 2260.239 171.000
## baseline.pvalue.scaled baseline.chisq.scaling.factor
## 0.000 1.239
## cfi tli
## 0.970 0.962
## nnfi rfi
## 0.962 0.903
## nfi pnfi
## 0.925 0.719
## ifi rni
## 0.971 0.970
## cfi.scaled tli.scaled
## 0.976 0.970
## cfi.robust tli.robust
## 0.978 0.972
## nnfi.scaled nnfi.robust
## 0.970 0.972
## rfi.scaled nfi.scaled
## 0.896 0.919
## ifi.scaled rni.scaled
## 0.977 0.976
## rni.robust logl
## 0.978 -3456.819
## unrestricted.logl aic
## -3351.128 7027.638
## bic ntotal
## 7200.744 154.000
## bic2 scaling.factor.h1
## 7020.331 1.191
## scaling.factor.h0 rmsea
## 1.269 0.062
## rmsea.ci.lower rmsea.ci.upper
## 0.046 0.077
## rmsea.pvalue rmsea.scaled
## 0.108 0.049
## rmsea.ci.lower.scaled rmsea.ci.upper.scaled
## 0.031 0.065
## rmsea.pvalue.scaled rmsea.robust
## 0.520 0.053
## rmsea.ci.lower.robust rmsea.ci.upper.robust
## 0.032 0.071
## rmsea.pvalue.robust rmr
## NA 0.056
## rmr_nomean srmr
## 0.056 0.038
## srmr_bentler srmr_bentler_nomean
## 0.038 0.038
## crmr crmr_nomean
## 0.040 0.040
## srmr_mplus srmr_mplus_nomean
## 0.038 0.038
## cn_05 cn_01
## 118.233 127.659
## gfi agfi
## 0.877 0.824
## pgfi mfi
## 0.614 0.775
## ecvi
## 2.113The results from the summary function indicate that the bifactor model fits the PCS data well, with robust model-fit statistics, CFI = .98, TLI = .97, RMSEA = .05. Thus, it is reasonable to calculate ωh to estimate how reliably the PCS total score measures the general psychological-cost factor.
reliability(fitBf)## gen s1 s2 s3 s4
## alpha 0.9638781 0.92504205 0.8992820 0.9052459 0.9405882
## omega 0.9741033 0.56377307 0.7884791 0.6766430 0.7816839
## omega2 0.9094893 0.09237594 0.3666293 0.1880759 0.2054075
## omega3 0.9077636 0.09240479 0.3666634 0.1878380 0.2053012
## avevar NA NA NA NA NAEstimates listed under the gen column pertain to the general cost factor. The omega estimate, .97, ignores the contribution of the specific factors to calculation of the implied variance of the total score in its denominator (Jorgensen et al., 2020); thus, this value is not a reliability estimate for the PCS total score. Instead, the omega2 and omega3 values under gen are ωh estimates; omega2 is calculated using the model-implied variance of the total score in its denominator, and omega3 is calculated using the observed sample variance of X (this distinction is analogous to the one between omega2 and omega3 described earlier for ωu). Thus, the proportion of PCS total-score variance that is due to a general psychological cost factor over and above the influence of effects that are specific to the different content domains is .91.
Unfortunately, ci.reliability cannot obtain a CI for the ωh estimate presented here.
Higher-order models
In this bifactor-model example, I considered multidimensionality among the PCS items a nuisance for the measurement of a broad, general psychological-cost target construct. In other situations, researchers may hypothesize an alternative multidimensional structure such that there is a broad, overarching construct indirectly influencing all items in a test through more conceptually narrow constructs that directly influence different subsets of items. Such hypotheses imply that the item-level data arise from a higher-order model, in which a higher-order factor (also known as a secondorder factor) causes individual differences in several more conceptually narrow lower-order factors (or firstorder factors), which in turn directly influence the observed item responses. In this context, researchers may evaluate the extent to which the test produces reliable total scores (as measures of the construct represented by the higher-order factor) as well as subscale scores (as measures of the constructs represented by the lower-order factors).
When item scores arise from a higher-order model, a reliability measure termed omega-higher-order, or ωho, represents the proportion of total-score variance that is due to the higher-order factor; parameter estimates from a higher-order model are used to calculate ωho.
#Specify the higher-order factor model for the PCS items
homod <- 'TE =~ TE1 + TE2 + TE3 + TE4 + TE5
OE =~ OE1 + OE2 + OE3 + OE4
LV =~ LVA1 + LVA2 + LVA3 + LVA4
EM =~ EM1 + EM2 + EM3 + EM4 + EM5 + EM6
cost =~ TE + OE + LV + EM'#Estimate the model and get the results
fitHo <- cfa(homod, data=pcs, std.lv=T, estimator='MLM')
summary(fitHo, fit.measures=T)## lavaan 0.6-9 ended normally after 57 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 42
##
## Used Total
## Number of observations 154 172
##
## Model Test User Model:
## Standard Robust
## Test Statistic 243.444 185.699
## Degrees of freedom 148 148
## P-value (Chi-square) 0.000 0.019
## Scaling correction factor 1.311
## Satorra-Bentler correction
##
## Model Test Baseline Model:
##
## Test statistic 2799.877 2282.637
## Degrees of freedom 171 171
## P-value 0.000 0.000
## Scaling correction factor 1.227
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.964 0.982
## Tucker-Lewis Index (TLI) 0.958 0.979
##
## Robust Comparative Fit Index (CFI) 0.981
## Robust Tucker-Lewis Index (TLI) 0.978
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -3472.850 -3472.850
## Loglikelihood unrestricted model (H1) -3351.128 -3351.128
##
## Akaike (AIC) 7029.700 7029.700
## Bayesian (BIC) 7157.252 7157.252
## Sample-size adjusted Bayesian (BIC) 7024.315 7024.315
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.065 0.041
## 90 Percent confidence interval - lower 0.050 0.021
## 90 Percent confidence interval - upper 0.079 0.056
## P-value RMSEA <= 0.05 0.052 0.832
##
## Robust RMSEA 0.047
## 90 Percent confidence interval - lower 0.020
## 90 Percent confidence interval - upper 0.066
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.045 0.045
##
## Parameter Estimates:
##
## Standard errors Robust.sem
## Information Expected
## Information saturated (h1) model Structured
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## TE =~
## TE1 0.356 0.067 5.289 0.000
## TE2 0.364 0.065 5.554 0.000
## TE3 0.299 0.059 5.056 0.000
## TE4 0.323 0.058 5.605 0.000
## TE5 0.338 0.065 5.198 0.000
## OE =~
## OE1 0.641 0.074 8.642 0.000
## OE2 0.616 0.060 10.334 0.000
## OE3 0.630 0.066 9.523 0.000
## OE4 0.647 0.064 10.078 0.000
## LV =~
## LVA1 0.442 0.056 7.836 0.000
## LVA2 0.457 0.059 7.751 0.000
## LVA3 0.427 0.059 7.286 0.000
## LVA4 0.460 0.056 8.258 0.000
## EM =~
## EM1 0.509 0.069 7.394 0.000
## EM2 0.462 0.066 7.018 0.000
## EM3 0.517 0.074 6.955 0.000
## EM4 0.483 0.067 7.179 0.000
## EM5 0.535 0.069 7.756 0.000
## EM6 0.595 0.080 7.430 0.000
## cost =~
## TE 2.914 0.595 4.894 0.000
## OE 1.223 0.155 7.880 0.000
## LV 1.951 0.281 6.933 0.000
## EM 1.900 0.327 5.809 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .TE1 0.321 0.063 5.081 0.000
## .TE2 0.473 0.069 6.837 0.000
## .TE3 0.531 0.100 5.289 0.000
## .TE4 0.398 0.073 5.494 0.000
## .TE5 0.418 0.070 5.935 0.000
## .OE1 0.531 0.099 5.353 0.000
## .OE2 0.440 0.081 5.428 0.000
## .OE3 0.495 0.099 5.001 0.000
## .OE4 0.315 0.055 5.719 0.000
## .LVA1 0.443 0.081 5.466 0.000
## .LVA2 0.170 0.035 4.851 0.000
## .LVA3 0.412 0.064 6.473 0.000
## .LVA4 0.533 0.090 5.945 0.000
## .EM1 0.637 0.085 7.508 0.000
## .EM2 0.367 0.062 5.872 0.000
## .EM3 0.493 0.075 6.564 0.000
## .EM4 0.387 0.064 6.017 0.000
## .EM5 0.315 0.062 5.059 0.000
## .EM6 0.554 0.085 6.515 0.000
## .TE 1.000
## .OE 1.000
## .LV 1.000
## .EM 1.000
## cost 1.000Obtaining omega-ho from a higher-order model for the Psychological Cost Scale
reliabilityL2(fitHo, 'cost')## omegaL1 omegaL2 partialOmegaL1
## 0.9088176 0.9410190 0.9734520#Obtain omega estimates for the subscale scores as measures of the lower-order factors
reliability(fitHo)## TE OE LV EM
## alpha 0.9250420 0.8992820 0.9052459 0.9405882
## omega 0.9260209 0.9000548 0.9077522 0.9415490
## omega2 0.9260209 0.9000548 0.9077522 0.9415490
## omega3 0.9256773 0.9014397 0.9125259 0.9404921
## avevar 0.7155347 0.6925123 0.7111716 0.7299181Exploratory Omega Estimates
The examples presented thus far used semTools:: reliability (or reliabilityL2) to estimate forms of omega from the results of CFA models. However, these estimates depend on correct specification of the model underlying a given test (e.g., ωu is not an appropriate reliability estimate if the population model is multidimensional, as evidenced by poor fit of a onefactor model). When no hypothesized CFA model fits the data (as may be particularly likely in the early stages of test development), EFA can be used to help uncover a test’s dimensional structure. Once the optimal number of factors underlying a test is determined, the omega function from the psych package (Revelle, 2020) can be used to obtain an omega estimate based on EFA model parameters; this omega estimate will represent the proportion of total-score variance due to a general factor common to all items (see the online supplement for further discussion and a demonstration of this omega function).
Using the psych package to calculate omega estimates
#Determine number of factors
library(parameters)
n_factors(pcs, package = "all")## # Method Agreement Procedure:
##
## The choice of 4 dimensions is supported by 7 (25.93%) methods out of 27 (beta, Scree (SE), EGA (glasso), Velicer's MAP, BIC, Fit_off, BIC).#Determine omega for 4 factor model
omega(pcs, nfactors = 4, plot = F)## Omega
## Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
## digits = digits, title = title, sl = sl, labels = labels,
## plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
## covar = covar)
## Alpha: 0.96
## G.6: 0.98
## Omega Hierarchical: 0.85
## Omega H asymptotic: 0.88
## Omega Total 0.98
##
## Schmid Leiman Factor loadings greater than 0.2
## g F1* F2* F3* F4* h2 u2 p2
## TE1 0.81 0.39 0.81 0.19 0.80
## TE2 0.76 0.40 0.75 0.25 0.77
## TE3 0.70 0.35 0.62 0.38 0.80
## TE4 0.78 0.29 0.72 0.28 0.84
## TE5 0.77 0.34 0.74 0.26 0.80
## OE1 0.61 0.55 0.67 0.33 0.56
## OE2 0.67 0.46 0.66 0.34 0.68
## OE3 0.61 0.54 0.68 0.32 0.54
## OE4 0.63 0.62 0.80 0.20 0.50
## LVA1 0.75 0.28 0.67 0.33 0.85
## LVA2 0.80 0.48 0.87 0.13 0.73
## LVA3 0.71 0.20 0.37 0.71 0.29 0.72
## LVA4 0.70 0.43 0.67 0.33 0.72
## EM1 0.69 0.39 0.66 0.34 0.73
## EM2 0.76 0.38 0.75 0.25 0.77
## EM3 0.72 0.47 0.73 0.27 0.71
## EM4 0.72 0.48 0.74 0.26 0.70
## EM5 0.78 0.43 0.81 0.19 0.76
## EM6 0.73 0.48 0.78 0.22 0.68
##
## With eigenvalues of:
## g F1* F2* F3* F4*
## 9.95 1.21 1.29 0.70 0.68
##
## general/max 7.74 max/min = 1.88
## mean percent general = 0.72 with sd = 0.1 and cv of 0.13
## Explained Common Variance of the general factor = 0.72
##
## The degrees of freedom are 101 and the fit is 1
## The number of observations was 172 with Chi Square = 161.11 with prob < 0.00013
## The root mean square of the residuals is 0.02
## The df corrected root mean square of the residuals is 0.03
## RMSEA index = 0.059 and the 10 % confidence intervals are 0.041 0.076
## BIC = -358.79
##
## Compare this with the adequacy of just a general factor and no group factors
## The degrees of freedom for just the general factor are 152 and the fit is 4.49
## The number of observations was 172 with Chi Square = 732.57 with prob < 8.2e-77
## The root mean square of the residuals is 0.1
## The df corrected root mean square of the residuals is 0.11
##
## RMSEA index = 0.149 and the 10 % confidence intervals are 0.139 0.16
## BIC = -49.85
##
## Measures of factor score adequacy
## g F1* F2* F3* F4*
## Correlation of scores with factors 0.93 0.79 0.86 0.72 0.77
## Multiple R square of scores with factors 0.86 0.63 0.73 0.52 0.59
## Minimum correlation of factor score estimates 0.73 0.26 0.47 0.04 0.18
##
## Total, General and Subset omega for each subset
## g F1* F2* F3* F4*
## Omega total for total scores and subscales 0.98 0.94 0.90 0.92 0.9
## Omega general for total scores and subscales 0.85 0.69 0.52 0.76 0.7
## Omega group for total scores and subscales 0.08 0.25 0.38 0.16 0.2Conclusions
Nevertheless, the extant studies clearly support a general preference for omega estimates over alpha (e.g., Trizano-Hermosilla & Alvarado, 2016; Yang & Green, 2010; Zinbarg et al., 2006). Yet, just as researchers should not blindly report alpha for the reliability of a test, they should not thoughtlessly report an omega coefficient without first investigating the test’s internal structure. The main conceptual benefit of using an omega coefficient to estimate reliability is realized when omega is based on a thoughtful modeling process that focuses on how well a test measures a target construct. Moving beyond mindless reliance on coefficient alpha and giving more careful attention to measurement quality is an important aspect of overcoming the replication crisis.
Daniel McNeish(2017). Thanks Coefficient Alpha, We’ll Take It From Here
library(psych)
library(MBESS)
library(userfriendlyscience)
library(coefficientalpha)
data(bfi, package = "psych")To simplify the analysis, we will separately break the full data into five separate data sets such that each of the five subscales are contained within their own data set.
agre<-bfi[,1:5]
cons<-bfi[,6:10]
extr<-bfi[,11:15]
neur<-bfi[,16:20]
open<-bfi[,21:25]Reverse Scoring
As is common in psychometric scales, some items may need to be reverse scored (this is required for appropriate calculation of some reliability coefficients like Cronbach’s alpha). This can be done with the invertItems function that is part of the userfriendlyscience package.
agreRev <- invertItems(agre, 1)
consRev <- invertItems(cons, c(4,5))
extrRev <- invertItems(extr, c(1, 2))
openRev <- invertItems(open, c(2, 5))Cronbach’s Alpha
coefficientalpha::alpha(agreRev)## Test of tau-equavilence
## The robust F statistic is 20.897
## with a p-value 0
## **The F test rejected tau-equavilence**
##
## The alpha is 0.7154321.
## About 20.11% of cases were downweighted.coefficientalpha::alpha(consRev)## Test of tau-equavilence
## The robust F statistic is 25.985
## with a p-value 0
## **The F test rejected tau-equavilence**
##
## The alpha is 0.7469157.
## About 16% of cases were downweighted.coefficientalpha::alpha(extrRev)## Test of tau-equavilence
## The robust F statistic is 27.013
## with a p-value 0
## **The F test rejected tau-equavilence**
##
## The alpha is 0.7795809.
## About 14.57% of cases were downweighted.coefficientalpha::alpha(neur)## Test of tau-equavilence
## The robust F statistic is 44.856
## with a p-value 0
## **The F test rejected tau-equavilence**
##
## The alpha is 0.8273471.
## About 11.25% of cases were downweighted.coefficientalpha::alpha(openRev)## Test of tau-equavilence
## The robust F statistic is 18.986
## with a p-value 0
## **The F test rejected tau-equavilence**
##
## The alpha is 0.6239177.
## About 17.29% of cases were downweighted.psych::alpha(agreRev)##
## Reliability analysis
## Call: psych::alpha(x = agreRev)
##
## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd median_r
## 0.7 0.71 0.68 0.33 2.5 0.009 4.7 0.9 0.34
##
## lower alpha upper 95% confidence boundaries
## 0.69 0.7 0.72
##
## Reliability if an item is dropped:
## raw_alpha std.alpha G6(smc) average_r S/N alpha se var.r med.r
## A1 0.72 0.73 0.67 0.40 2.6 0.0087 0.0065 0.38
## A2 0.62 0.63 0.58 0.29 1.7 0.0119 0.0169 0.29
## A3 0.60 0.61 0.56 0.28 1.6 0.0124 0.0094 0.32
## A4 0.69 0.69 0.65 0.36 2.3 0.0098 0.0159 0.37
## A5 0.64 0.66 0.61 0.32 1.9 0.0111 0.0126 0.34
##
## Item statistics
## n raw.r std.r r.cor r.drop mean sd
## A1 2784 0.58 0.57 0.38 0.31 4.6 1.4
## A2 2773 0.73 0.75 0.67 0.56 4.8 1.2
## A3 2774 0.76 0.77 0.71 0.59 4.6 1.3
## A4 2781 0.65 0.63 0.47 0.39 4.7 1.5
## A5 2784 0.69 0.70 0.60 0.49 4.6 1.3
##
## Non missing response frequency for each item
## 1 2 3 4 5 6 miss
## A1 0.03 0.08 0.12 0.14 0.29 0.33 0.01
## A2 0.02 0.05 0.05 0.20 0.37 0.31 0.01
## A3 0.03 0.06 0.07 0.20 0.36 0.27 0.01
## A4 0.05 0.08 0.07 0.16 0.24 0.41 0.01
## A5 0.02 0.07 0.09 0.22 0.35 0.25 0.01psych::alpha(consRev)##
## Reliability analysis
## Call: psych::alpha(x = consRev)
##
## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd median_r
## 0.73 0.73 0.69 0.35 2.7 0.0081 4.3 0.95 0.34
##
## lower alpha upper 95% confidence boundaries
## 0.71 0.73 0.74
##
## Reliability if an item is dropped:
## raw_alpha std.alpha G6(smc) average_r S/N alpha se var.r med.r
## C1 0.69 0.70 0.64 0.36 2.3 0.0093 0.0037 0.35
## C2 0.67 0.67 0.62 0.34 2.1 0.0099 0.0056 0.34
## C3 0.69 0.69 0.64 0.36 2.3 0.0096 0.0070 0.36
## C4 0.65 0.66 0.60 0.33 2.0 0.0107 0.0037 0.32
## C5 0.69 0.69 0.63 0.36 2.2 0.0096 0.0017 0.35
##
## Item statistics
## n raw.r std.r r.cor r.drop mean sd
## C1 2779 0.65 0.67 0.54 0.45 4.5 1.2
## C2 2776 0.70 0.71 0.60 0.50 4.4 1.3
## C3 2780 0.66 0.67 0.54 0.46 4.3 1.3
## C4 2774 0.74 0.73 0.64 0.55 4.4 1.4
## C5 2784 0.72 0.68 0.57 0.48 3.7 1.6
##
## Non missing response frequency for each item
## 1 2 3 4 5 6 miss
## C1 0.03 0.06 0.10 0.24 0.37 0.21 0.01
## C2 0.03 0.09 0.11 0.23 0.35 0.20 0.01
## C3 0.03 0.09 0.11 0.27 0.34 0.17 0.01
## C4 0.02 0.08 0.16 0.17 0.29 0.28 0.01
## C5 0.10 0.17 0.22 0.12 0.20 0.18 0.01psych::alpha(extrRev)##
## Reliability analysis
## Call: psych::alpha(x = extrRev)
##
## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd median_r
## 0.76 0.76 0.73 0.39 3.2 0.007 4.1 1.1 0.38
##
## lower alpha upper 95% confidence boundaries
## 0.75 0.76 0.78
##
## Reliability if an item is dropped:
## raw_alpha std.alpha G6(smc) average_r S/N alpha se var.r med.r
## E1 0.73 0.73 0.67 0.40 2.6 0.0084 0.0044 0.38
## E2 0.69 0.69 0.63 0.36 2.3 0.0095 0.0028 0.35
## E3 0.73 0.73 0.67 0.40 2.7 0.0082 0.0071 0.40
## E4 0.70 0.70 0.65 0.37 2.4 0.0091 0.0033 0.38
## E5 0.74 0.74 0.69 0.42 2.9 0.0078 0.0043 0.42
##
## Item statistics
## n raw.r std.r r.cor r.drop mean sd
## E1 2777 0.72 0.70 0.59 0.52 4.0 1.6
## E2 2784 0.78 0.76 0.69 0.61 3.9 1.6
## E3 2775 0.68 0.70 0.58 0.50 4.0 1.4
## E4 2791 0.75 0.75 0.66 0.58 4.4 1.5
## E5 2779 0.64 0.66 0.52 0.45 4.4 1.3
##
## Non missing response frequency for each item
## 1 2 3 4 5 6 miss
## E1 0.09 0.13 0.16 0.15 0.23 0.24 0.01
## E2 0.09 0.14 0.22 0.12 0.24 0.19 0.01
## E3 0.05 0.11 0.15 0.30 0.27 0.13 0.01
## E4 0.05 0.09 0.10 0.16 0.34 0.26 0.00
## E5 0.03 0.08 0.10 0.22 0.34 0.22 0.01psych::alpha(neur)##
## Reliability analysis
## Call: psych::alpha(x = neur)
##
## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd median_r
## 0.81 0.81 0.8 0.47 4.4 0.0056 3.2 1.2 0.41
##
## lower alpha upper 95% confidence boundaries
## 0.8 0.81 0.82
##
## Reliability if an item is dropped:
## raw_alpha std.alpha G6(smc) average_r S/N alpha se var.r med.r
## N1 0.76 0.76 0.71 0.44 3.1 0.0075 0.0061 0.41
## N2 0.76 0.76 0.72 0.45 3.2 0.0073 0.0054 0.41
## N3 0.76 0.76 0.73 0.44 3.1 0.0077 0.0179 0.39
## N4 0.80 0.80 0.77 0.50 3.9 0.0064 0.0182 0.49
## N5 0.81 0.81 0.79 0.52 4.3 0.0059 0.0137 0.53
##
## Item statistics
## n raw.r std.r r.cor r.drop mean sd
## N1 2778 0.80 0.80 0.76 0.67 2.9 1.6
## N2 2779 0.79 0.79 0.75 0.65 3.5 1.5
## N3 2789 0.81 0.81 0.74 0.67 3.2 1.6
## N4 2764 0.72 0.71 0.60 0.54 3.2 1.6
## N5 2771 0.68 0.67 0.53 0.49 3.0 1.6
##
## Non missing response frequency for each item
## 1 2 3 4 5 6 miss
## N1 0.24 0.24 0.15 0.19 0.12 0.07 0.01
## N2 0.12 0.19 0.15 0.26 0.18 0.10 0.01
## N3 0.18 0.23 0.13 0.21 0.16 0.09 0.00
## N4 0.17 0.24 0.15 0.22 0.14 0.09 0.01
## N5 0.24 0.24 0.14 0.18 0.12 0.09 0.01psych::alpha(openRev)##
## Reliability analysis
## Call: psych::alpha(x = openRev)
##
## raw_alpha std.alpha G6(smc) average_r S/N ase mean sd median_r
## 0.6 0.61 0.57 0.24 1.5 0.012 4.6 0.81 0.23
##
## lower alpha upper 95% confidence boundaries
## 0.58 0.6 0.62
##
## Reliability if an item is dropped:
## raw_alpha std.alpha G6(smc) average_r S/N alpha se var.r med.r
## O1 0.53 0.53 0.48 0.22 1.1 0.014 0.0092 0.23
## O2 0.57 0.57 0.51 0.25 1.3 0.013 0.0076 0.22
## O3 0.50 0.50 0.44 0.20 1.0 0.015 0.0071 0.20
## O4 0.61 0.62 0.56 0.29 1.6 0.012 0.0044 0.29
## O5 0.51 0.53 0.47 0.22 1.1 0.015 0.0115 0.20
##
## Item statistics
## n raw.r std.r r.cor r.drop mean sd
## O1 2778 0.62 0.65 0.52 0.39 4.8 1.1
## O2 2800 0.65 0.60 0.43 0.33 4.3 1.6
## O3 2772 0.67 0.69 0.59 0.45 4.4 1.2
## O4 2786 0.50 0.52 0.29 0.22 4.9 1.2
## O5 2780 0.67 0.66 0.52 0.42 4.5 1.3
##
## Non missing response frequency for each item
## 1 2 3 4 5 6 miss
## O1 0.01 0.04 0.08 0.22 0.33 0.33 0.01
## O2 0.06 0.10 0.16 0.14 0.26 0.29 0.00
## O3 0.03 0.05 0.11 0.28 0.34 0.20 0.01
## O4 0.02 0.04 0.06 0.17 0.32 0.39 0.01
## O5 0.03 0.07 0.13 0.19 0.32 0.27 0.01Omega Total
library(MBESS)
ci.reliability(agreRev)## $est
## [1] 0.7104129
##
## $se
## [1] 0.01018984
##
## $ci.lower
## [1] 0.6904412
##
## $ci.upper
## [1] 0.7303846
##
## $conf.level
## [1] 0.95
##
## $type
## [1] "omega"
##
## $interval.type
## [1] "robust maximum likelihood (wald ci)"Revelle’s Omega Total
psych::omega(agreRev)## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Warning in cov2cor(t(w) %*% r %*% w): diag(.) had 0 or NA entries; non-finite
## result is doubtful
## Omega
## Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
## digits = digits, title = title, sl = sl, labels = labels,
## plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
## covar = covar)
## Alpha: 0.71
## G.6: 0.68
## Omega Hierarchical: 0.68
## Omega H asymptotic: 0.9
## Omega Total 0.76
##
## Schmid Leiman Factor loadings greater than 0.2
## g F1* F2* F3* h2 u2 p2
## A1 0.39 0.46 0.36 0.64 0.41
## A2 0.71 0.51 0.49 0.98
## A3 0.66 0.43 0.61 0.39 0.71
## A4 0.51 0.26 0.74 0.98
## A5 0.55 0.33 0.42 0.58 0.72
##
## With eigenvalues of:
## g F1* F2* F3*
## 1.65 0.29 0.24 0.00
##
## general/max 5.67 max/min = Inf
## mean percent general = 0.76 with sd = 0.24 and cv of 0.31
## Explained Common Variance of the general factor = 0.76
##
## The degrees of freedom are -2 and the fit is 0
## The number of observations was 2800 with Chi Square = 0 with prob < NA
## The root mean square of the residuals is 0
## The df corrected root mean square of the residuals is NA
##
## Compare this with the adequacy of just a general factor and no group factors
## The degrees of freedom for just the general factor are 5 and the fit is 0.06
## The number of observations was 2800 with Chi Square = 158.43 with prob < 2.1e-32
## The root mean square of the residuals is 0.05
## The df corrected root mean square of the residuals is 0.08
##
## RMSEA index = 0.105 and the 10 % confidence intervals are 0.091 0.119
## BIC = 118.74
##
## Measures of factor score adequacy
## g F1* F2* F3*
## Correlation of scores with factors 0.84 0.53 0.51 0
## Multiple R square of scores with factors 0.71 0.28 0.26 0
## Minimum correlation of factor score estimates 0.42 -0.45 -0.48 -1
##
## Total, General and Subset omega for each subset
## g F1* F2* F3*
## Omega total for total scores and subscales 0.76 0.68 0.60 NA
## Omega general for total scores and subscales 0.68 0.49 0.55 NA
## Omega group for total scores and subscales 0.07 0.19 0.04 NApsych::omega(consRev)## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Warning in cov2cor(t(w) %*% r %*% w): diag(.) had 0 or NA entries; non-finite
## result is doubtful
## Omega
## Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
## digits = digits, title = title, sl = sl, labels = labels,
## plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
## covar = covar)
## Alpha: 0.73
## G.6: 0.69
## Omega Hierarchical: 0.61
## Omega H asymptotic: 0.8
## Omega Total 0.77
##
## Schmid Leiman Factor loadings greater than 0.2
## g F1* F2* F3* h2 u2 p2
## C1 0.40 0.47 0.39 0.61 0.40
## C2 0.45 0.52 0.47 0.53 0.43
## C3 0.45 0.34 0.21 0.33 0.67 0.59
## C4 0.72 0.52 0.48 1.00
## C5 0.70 0.51 0.49 0.96
##
## With eigenvalues of:
## g F1* F2* F3*
## 1.57 0.61 0.00 0.08
##
## general/max 2.59 max/min = Inf
## mean percent general = 0.68 with sd = 0.29 and cv of 0.42
## Explained Common Variance of the general factor = 0.7
##
## The degrees of freedom are -2 and the fit is 0
## The number of observations was 2800 with Chi Square = 0 with prob < NA
## The root mean square of the residuals is 0
## The df corrected root mean square of the residuals is NA
##
## Compare this with the adequacy of just a general factor and no group factors
## The degrees of freedom for just the general factor are 5 and the fit is 0.13
## The number of observations was 2800 with Chi Square = 352.1 with prob < 6.2e-74
## The root mean square of the residuals is 0.11
## The df corrected root mean square of the residuals is 0.15
##
## RMSEA index = 0.157 and the 10 % confidence intervals are 0.144 0.172
## BIC = 312.42
##
## Measures of factor score adequacy
## g F1* F2* F3*
## Correlation of scores with factors 0.85 0.67 0 0.35
## Multiple R square of scores with factors 0.72 0.45 0 0.12
## Minimum correlation of factor score estimates 0.44 -0.10 -1 -0.76
##
## Total, General and Subset omega for each subset
## g F1* F2* F3*
## Omega total for total scores and subscales 0.77 0.66 NA 0.68
## Omega general for total scores and subscales 0.61 0.32 NA 0.68
## Omega group for total scores and subscales 0.15 0.34 NA 0.00psych::omega(extrRev)
## Omega
## Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
## digits = digits, title = title, sl = sl, labels = labels,
## plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
## covar = covar)
## Alpha: 0.76
## G.6: 0.73
## Omega Hierarchical: 0.66
## Omega H asymptotic: 0.83
## Omega Total 0.79
##
## Schmid Leiman Factor loadings greater than 0.2
## g F1* F2* F3* h2 u2 p2
## E1 0.49 0.36 0.37 0.63 0.65
## E2 0.59 0.50 0.60 0.40 0.57
## E3 0.66 0.47 0.53 0.92
## E4 0.58 0.38 0.22 0.53 0.47 0.64
## E5 0.59 0.40 0.60 0.87
##
## With eigenvalues of:
## g F1* F2* F3*
## 1.70 0.53 0.04 0.11
##
## general/max 3.22 max/min = 13.29
## mean percent general = 0.73 with sd = 0.15 and cv of 0.21
## Explained Common Variance of the general factor = 0.72
##
## The degrees of freedom are -2 and the fit is 0
## The number of observations was 2800 with Chi Square = 0 with prob < NA
## The root mean square of the residuals is 0
## The df corrected root mean square of the residuals is NA
##
## Compare this with the adequacy of just a general factor and no group factors
## The degrees of freedom for just the general factor are 5 and the fit is 0.11
## The number of observations was 2800 with Chi Square = 312.5 with prob < 2.1e-65
## The root mean square of the residuals is 0.09
## The df corrected root mean square of the residuals is 0.13
##
## RMSEA index = 0.148 and the 10 % confidence intervals are 0.135 0.162
## BIC = 272.81
##
## Measures of factor score adequacy
## g F1* F2* F3*
## Correlation of scores with factors 0.83 0.61 0.20 0.41
## Multiple R square of scores with factors 0.68 0.38 0.04 0.17
## Minimum correlation of factor score estimates 0.37 -0.25 -0.92 -0.66
##
## Total, General and Subset omega for each subset
## g F1* F2* F3*
## Omega total for total scores and subscales 0.79 0.74 0.45 0.38
## Omega general for total scores and subscales 0.66 0.47 0.43 0.35
## Omega group for total scores and subscales 0.13 0.27 0.02 0.03psych::omega(neur)
## Omega
## Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
## digits = digits, title = title, sl = sl, labels = labels,
## plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
## covar = covar)
## Alpha: 0.81
## G.6: 0.8
## Omega Hierarchical: 0.73
## Omega H asymptotic: 0.86
## Omega Total 0.85
##
## Schmid Leiman Factor loadings greater than 0.2
## g F1* F2* F3* h2 u2 p2
## N1 0.61 0.59 0.73 0.27 0.51
## N2 0.60 0.58 0.70 0.30 0.51
## N3 0.75 0.59 0.41 0.95
## N4 0.70 0.50 0.50 0.99
## N5 0.57 0.35 0.65 0.92
##
## With eigenvalues of:
## g F1* F2* F3*
## 2.11 0.72 0.01 0.03
##
## general/max 2.94 max/min = 129.98
## mean percent general = 0.78 with sd = 0.24 and cv of 0.31
## Explained Common Variance of the general factor = 0.74
##
## The degrees of freedom are -2 and the fit is 0
## The number of observations was 2800 with Chi Square = 0 with prob < NA
## The root mean square of the residuals is 0
## The df corrected root mean square of the residuals is NA
##
## Compare this with the adequacy of just a general factor and no group factors
## The degrees of freedom for just the general factor are 5 and the fit is 0.31
## The number of observations was 2800 with Chi Square = 858.38 with prob < 2.7e-183
## The root mean square of the residuals is 0.12
## The df corrected root mean square of the residuals is 0.17
##
## RMSEA index = 0.247 and the 10 % confidence intervals are 0.233 0.261
## BIC = 818.69
##
## Measures of factor score adequacy
## g F1* F2* F3*
## Correlation of scores with factors 0.87 0.75 0.07 0.25
## Multiple R square of scores with factors 0.76 0.57 0.00 0.06
## Minimum correlation of factor score estimates 0.53 0.13 -0.99 -0.87
##
## Total, General and Subset omega for each subset
## g F1* F2* F3*
## Omega total for total scores and subscales 0.85 0.85 NA 0.35
## Omega general for total scores and subscales 0.73 0.69 NA 0.33
## Omega group for total scores and subscales 0.12 0.16 NA 0.02psych::omega(openRev)## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Warning in cov2cor(t(w) %*% r %*% w): diag(.) had 0 or NA entries; non-finite
## result is doubtful
## Omega
## Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
## digits = digits, title = title, sl = sl, labels = labels,
## plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
## covar = covar)
## Alpha: 0.61
## G.6: 0.57
## Omega Hierarchical: 0.51
## Omega H asymptotic: 0.77
## Omega Total 0.67
##
## Schmid Leiman Factor loadings greater than 0.2
## g F1* F2* F3* h2 u2 p2
## O1 0.35 0.53 0.41 0.59 0.31
## O2 0.46 -0.35 0.35 0.65 0.61
## O3 0.47 0.43 0.41 0.59 0.54
## O4 0.30 0.24 0.16 0.84 0.56
## O5 0.65 0.42 0.58 1.00
##
## With eigenvalues of:
## g F1* F2* F3*
## 1.07 0.00 0.50 0.18
##
## general/max 2.16 max/min = Inf
## mean percent general = 0.6 with sd = 0.25 and cv of 0.41
## Explained Common Variance of the general factor = 0.61
##
## The degrees of freedom are -2 and the fit is 0
## The number of observations was 2800 with Chi Square = 0 with prob < NA
## The root mean square of the residuals is 0
## The df corrected root mean square of the residuals is NA
##
## Compare this with the adequacy of just a general factor and no group factors
## The degrees of freedom for just the general factor are 5 and the fit is 0.08
## The number of observations was 2800 with Chi Square = 232.87 with prob < 2.6e-48
## The root mean square of the residuals is 0.08
## The df corrected root mean square of the residuals is 0.12
##
## RMSEA index = 0.128 and the 10 % confidence intervals are 0.114 0.142
## BIC = 193.18
##
## Measures of factor score adequacy
## g F1* F2* F3*
## Correlation of scores with factors 0.76 0 0.62 0.44
## Multiple R square of scores with factors 0.58 0 0.38 0.19
## Minimum correlation of factor score estimates 0.16 -1 -0.24 -0.61
##
## Total, General and Subset omega for each subset
## g F1* F2* F3*
## Omega total for total scores and subscales 0.67 NA 0.57 0.49
## Omega general for total scores and subscales 0.51 NA 0.24 0.48
## Omega group for total scores and subscales 0.10 NA 0.33 0.01A convenient option in the omega function is that a polychoric covariance matrix can be estimated and used internally and is possible by specifying only two additional words in the code.
psych::omega(agreRev, poly=TRUE)## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Warning in cov2cor(t(w) %*% r %*% w): diag(.) had 0 or NA entries; non-finite
## result is doubtful
## Omega
## Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
## digits = digits, title = title, sl = sl, labels = labels,
## plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
## covar = covar)
## Alpha: 0.76
## G.6: 0.74
## Omega Hierarchical: 0.72
## Omega H asymptotic: 0.89
## Omega Total 0.8
##
## Schmid Leiman Factor loadings greater than 0.2
## g F1* F2* F3* h2 u2 p2
## A1 0.44 0.43 0.39 0.61 0.50
## A2 0.79 0.62 0.38 1.00
## A3 0.68 0.47 0.68 0.32 0.68
## A4 0.54 0.31 0.69 0.96
## A5 0.57 0.41 0.50 0.50 0.65
##
## With eigenvalues of:
## g F1* F2* F3*
## 1.89 0.39 0.23 0.00
##
## general/max 4.88 max/min = Inf
## mean percent general = 0.76 with sd = 0.21 and cv of 0.28
## Explained Common Variance of the general factor = 0.75
##
## The degrees of freedom are -2 and the fit is 0
## The number of observations was 2800 with Chi Square = 0 with prob < NA
## The root mean square of the residuals is 0
## The df corrected root mean square of the residuals is NA
##
## Compare this with the adequacy of just a general factor and no group factors
## The degrees of freedom for just the general factor are 5 and the fit is 0.1
## The number of observations was 2800 with Chi Square = 291.51 with prob < 6.7e-61
## The root mean square of the residuals is 0.07
## The df corrected root mean square of the residuals is 0.1
##
## RMSEA index = 0.143 and the 10 % confidence intervals are 0.129 0.157
## BIC = 251.82
##
## Measures of factor score adequacy
## g F1* F2* F3*
## Correlation of scores with factors 0.87 0.62 0.51 0
## Multiple R square of scores with factors 0.76 0.38 0.26 0
## Minimum correlation of factor score estimates 0.52 -0.24 -0.48 -1
##
## Total, General and Subset omega for each subset
## g F1* F2* F3*
## Omega total for total scores and subscales 0.80 0.74 0.66 NA
## Omega general for total scores and subscales 0.72 0.49 0.63 NA
## Omega group for total scores and subscales 0.07 0.24 0.03 NApsych::omega(consRev, poly=TRUE)## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Warning in cov2cor(t(w) %*% r %*% w): diag(.) had 0 or NA entries; non-finite
## result is doubtful
## Omega
## Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
## digits = digits, title = title, sl = sl, labels = labels,
## plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
## covar = covar)
## Alpha: 0.77
## G.6: 0.74
## Omega Hierarchical: 0.65
## Omega H asymptotic: 0.81
## Omega Total 0.81
##
## Schmid Leiman Factor loadings greater than 0.2
## g F1* F2* F3* h2 u2 p2
## C1 0.45 0.47 0.44 0.56 0.45
## C2 0.48 0.55 0.53 0.47 0.44
## C3 0.48 0.35 0.22 0.37 0.63 0.61
## C4 0.76 0.59 0.41 0.99
## C5 0.74 0.57 0.43 0.95
##
## With eigenvalues of:
## g F1* F2* F3*
## 1.78 0.65 0.00 0.09
##
## general/max 2.73 max/min = Inf
## mean percent general = 0.69 with sd = 0.27 and cv of 0.39
## Explained Common Variance of the general factor = 0.71
##
## The degrees of freedom are -2 and the fit is 0
## The number of observations was 2800 with Chi Square = 0 with prob < NA
## The root mean square of the residuals is 0
## The df corrected root mean square of the residuals is NA
##
## Compare this with the adequacy of just a general factor and no group factors
## The degrees of freedom for just the general factor are 5 and the fit is 0.16
## The number of observations was 2800 with Chi Square = 449.93 with prob < 5.1e-95
## The root mean square of the residuals is 0.11
## The df corrected root mean square of the residuals is 0.16
##
## RMSEA index = 0.178 and the 10 % confidence intervals are 0.165 0.192
## BIC = 410.24
##
## Measures of factor score adequacy
## g F1* F2* F3*
## Correlation of scores with factors 0.87 0.70 0 0.39
## Multiple R square of scores with factors 0.76 0.49 0 0.15
## Minimum correlation of factor score estimates 0.52 -0.02 -1 -0.70
##
## Total, General and Subset omega for each subset
## g F1* F2* F3*
## Omega total for total scores and subscales 0.81 0.71 NA 0.73
## Omega general for total scores and subscales 0.65 0.36 NA 0.73
## Omega group for total scores and subscales 0.15 0.34 NA 0.00psych::omega(extrRev, poly=TRUE)
## Omega
## Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
## digits = digits, title = title, sl = sl, labels = labels,
## plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
## covar = covar)
## Alpha: 0.79
## G.6: 0.76
## Omega Hierarchical: 0.77
## Omega H asymptotic: 0.93
## Omega Total 0.83
##
## Schmid Leiman Factor loadings greater than 0.2
## g F1* F2* F3* h2 u2 p2
## E1 0.65 0.42 0.58 0.99
## E2 0.79 0.63 0.37 0.98
## E3 0.55 0.45 0.53 0.47 0.58
## E4 0.76 0.21 0.62 0.38 0.92
## E5 0.50 0.41 -0.21 0.47 0.53 0.54
##
## With eigenvalues of:
## g F1* F2* F3*
## 2.16 0.01 0.37 0.12
##
## general/max 5.8 max/min = 60.87
## mean percent general = 0.8 with sd = 0.22 and cv of 0.28
## Explained Common Variance of the general factor = 0.81
##
## The degrees of freedom are -2 and the fit is 0
## The number of observations was 2800 with Chi Square = 0 with prob < NA
## The root mean square of the residuals is 0
## The df corrected root mean square of the residuals is NA
##
## Compare this with the adequacy of just a general factor and no group factors
## The degrees of freedom for just the general factor are 5 and the fit is 0.06
## The number of observations was 2800 with Chi Square = 158.72 with prob < 1.9e-32
## The root mean square of the residuals is 0.05
## The df corrected root mean square of the residuals is 0.08
##
## RMSEA index = 0.105 and the 10 % confidence intervals are 0.091 0.119
## BIC = 119.03
##
## Measures of factor score adequacy
## g F1* F2* F3*
## Correlation of scores with factors 0.90 0.06 0.61 0.47
## Multiple R square of scores with factors 0.81 0.00 0.37 0.22
## Minimum correlation of factor score estimates 0.63 -0.99 -0.26 -0.57
##
## Total, General and Subset omega for each subset
## g F1* F2* F3*
## Omega total for total scores and subscales 0.83 NA 0.65 0.78
## Omega general for total scores and subscales 0.77 NA 0.39 0.78
## Omega group for total scores and subscales 0.05 NA 0.26 0.00psych::omega(neur, poly=TRUE)
## Omega
## Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
## digits = digits, title = title, sl = sl, labels = labels,
## plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
## covar = covar)
## Alpha: 0.84
## G.6: 0.83
## Omega Hierarchical: 0.75
## Omega H asymptotic: 0.85
## Omega Total 0.88
##
## Schmid Leiman Factor loadings greater than 0.2
## g F1* F2* F3* h2 u2 p2
## N1 0.64 0.62 0.79 0.21 0.52
## N2 0.62 0.60 0.76 0.24 0.51
## N3 0.77 0.64 0.36 0.94
## N4 0.74 0.55 0.45 0.99
## N5 0.61 0.40 0.60 0.92
##
## With eigenvalues of:
## g F1* F2* F3*
## 2.30 0.78 0.01 0.04
##
## general/max 2.95 max/min = 123.44
## mean percent general = 0.78 with sd = 0.24 and cv of 0.31
## Explained Common Variance of the general factor = 0.74
##
## The degrees of freedom are -2 and the fit is 0
## The number of observations was 2800 with Chi Square = 0 with prob < NA
## The root mean square of the residuals is 0
## The df corrected root mean square of the residuals is NA
##
## Compare this with the adequacy of just a general factor and no group factors
## The degrees of freedom for just the general factor are 5 and the fit is 0.43
## The number of observations was 2800 with Chi Square = 1189.28 with prob < 6.2e-255
## The root mean square of the residuals is 0.13
## The df corrected root mean square of the residuals is 0.18
##
## RMSEA index = 0.291 and the 10 % confidence intervals are 0.277 0.305
## BIC = 1149.59
##
## Measures of factor score adequacy
## g F1* F2* F3*
## Correlation of scores with factors 0.89 0.79 0.07 0.30
## Multiple R square of scores with factors 0.79 0.62 0.01 0.09
## Minimum correlation of factor score estimates 0.58 0.24 -0.99 -0.82
##
## Total, General and Subset omega for each subset
## g F1* F2* F3*
## Omega total for total scores and subscales 0.88 0.88 0.54 0.39
## Omega general for total scores and subscales 0.75 0.60 0.54 0.37
## Omega group for total scores and subscales 0.13 0.28 0.00 0.03psych::omega(openRev, poly=TRUE)
## Omega
## Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
## digits = digits, title = title, sl = sl, labels = labels,
## plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
## covar = covar)
## Alpha: 0.67
## G.6: 0.64
## Omega Hierarchical: 0.61
## Omega H asymptotic: 0.84
## Omega Total 0.73
##
## Schmid Leiman Factor loadings greater than 0.2
## g F1* F2* F3* h2 u2 p2
## O1 0.69 0.48 0.52 0.99
## O2 0.42 0.37 -0.27 0.39 0.61 0.44
## O3 0.66 0.45 0.55 0.96
## O4 0.35 0.33 0.25 0.75 0.49
## O5 0.46 0.53 0.49 0.51 0.44
##
## With eigenvalues of:
## g F1* F2* F3*
## 1.42 0.45 0.00 0.19
##
## general/max 3.18 max/min = 227.41
## mean percent general = 0.66 with sd = 0.28 and cv of 0.43
## Explained Common Variance of the general factor = 0.69
##
## The degrees of freedom are -2 and the fit is 0
## The number of observations was 2800 with Chi Square = 0 with prob < NA
## The root mean square of the residuals is 0
## The df corrected root mean square of the residuals is NA
##
## Compare this with the adequacy of just a general factor and no group factors
## The degrees of freedom for just the general factor are 5 and the fit is 0.07
## The number of observations was 2800 with Chi Square = 193.63 with prob < 6.5e-40
## The root mean square of the residuals is 0.07
## The df corrected root mean square of the residuals is 0.1
##
## RMSEA index = 0.116 and the 10 % confidence intervals are 0.102 0.13
## BIC = 153.94
##
## Measures of factor score adequacy
## g F1* F2* F3*
## Correlation of scores with factors 0.83 0.61 0.04 0.46
## Multiple R square of scores with factors 0.68 0.38 0.00 0.21
## Minimum correlation of factor score estimates 0.36 -0.25 -1.00 -0.57
##
## Total, General and Subset omega for each subset
## g F1* F2* F3*
## Omega total for total scores and subscales 0.73 0.73 NA 0.23
## Omega general for total scores and subscales 0.61 0.61 NA 0.12
## Omega group for total scores and subscales 0.10 0.12 NA 0.11Greatest Lower Bound
The glb.fa function in the psych package estimates the greatest lower bound. Similar to other methods in the psych package, the only necessary argument of the function is the data name.
glb.fa(agreRev)## $glb
## [1] 0.7469251
##
## $communality
## A1 A2 A3 A4 A5
## 0.1936286 0.6103506 0.5731462 0.2258019 0.4534677
##
## $numf
## [1] 2
##
## $Call
## glb.fa(r = agreRev)glb.fa(consRev)## $glb
## [1] 0.7738612
##
## $communality
## C1 C2 C3 C4 C5
## 0.3825266 0.4787617 0.2871087 0.4132925 0.7194580
##
## $numf
## [1] 2
##
## $Call
## glb.fa(r = consRev)glb.fa(extrRev)## $glb
## [1] 0.8182759
##
## $communality
## E1 E2 E3 E4 E5
## 0.3806384 0.5938087 0.9956216 0.4509084 0.2534379
##
## $numf
## [1] 2
##
## $Call
## glb.fa(r = extrRev)glb.fa(neur)## $glb
## [1] 0.8486629
##
## $communality
## N1 N2 N3 N4 N5
## 0.7172433 0.6970850 0.5879636 0.5052678 0.3203767
##
## $numf
## [1] 2
##
## $Call
## glb.fa(r = neur)glb.fa(openRev)## $glb
## [1] 0.7034328
##
## $communality
## O1 O2 O3 O4 O5
## 0.3196781 0.9956973 0.4551818 0.1048350 0.2387815
##
## $numf
## [1] 2
##
## $Call
## glb.fa(r = openRev)Unfortunately, the glb.fa function does not offer the option to use a polychoric covariance matrix internally and therefore uses a Pearson covariance matrix. However, this can be circumvented by separately estimating a polychoric covariance or correlation matrix, and using that as the input file instead of the raw data. However, it can be a bit tricky to save a polychoric correlation matrix as a data frame in R. First, the polychoric matrix is estimated with the polychoric function from the psych package. Rather than immediately outputting the results, the output is saved to an object (called mat in the code below). The output contains both the polychoric correlation matrix and thresholds; the thresholds are not needed, so we want to exclude them and only save the matrix. In doing so, we also must convert the object to a data frame.
The R code for doing so for the agreeableness subscale is as follows:
mat<-polychoric(agreRev)
agre.poly<-as.data.frame(mat$rho)The glb.fa function can accept a correlation matrix as input, so we can use the saved polychoric correlation matrix as the input of the function.
glb.fa(agre.poly) #This will provide the desired output## $glb
## [1] 0.8112357
##
## $communality
## A1 A2 A3 A4 A5
## 0.1799552 0.9954731 0.6813344 0.2535902 0.4868817
##
## $numf
## [1] 2
##
## $Call
## glb.fa(r = agre.poly)scaleStructure Function
Although the above analyses are not difficult to perform because the commands are quite straightforward, for inexperienced or reluctant R users, the scaleStructure package can estimate these quantities in a single pass and summarizes the output.
scaleStructure(dat=agreRev, ci=FALSE)##
## Information about this analysis:
##
## Dataframe: agreRev
## Items: all
## Observations: 2709
## Positive correlations: 10 out of 10 (100%)
##
## Estimates assuming interval level:
##
## Omega (total): 0.71
## Omega (hierarchical): 0.68
## Revelle's omega (total): 0.76
## Greatest Lower Bound (GLB): 0.75
## Coefficient H: 0.77
## Cronbach's alpha: 0.7
##
## Estimates assuming ordinal level:
##
## Ordinal Omega (total): 0.77
## Ordinal Omega (hierarch.): 0.77
## Ordinal Cronbach's alpha: 0.76
##
## Note: the normal point estimate and confidence interval for omega are based on the procedure suggested by Dunn, Baguley & Brunsden (2013) using the MBESS function ci.reliability, whereas the psych package point estimate was suggested in Revelle & Zinbarg (2008). See the help ('?scaleStructure') for more information.scaleStructure(dat=consRev, ci=FALSE)##
## Information about this analysis:
##
## Dataframe: consRev
## Items: all
## Observations: 2707
## Positive correlations: 10 out of 10 (100%)
##
## Estimates assuming interval level:
##
## Omega (total): 0.73
## Omega (hierarchical): 0.62
## Revelle's omega (total): 0.77
## Greatest Lower Bound (GLB): 0.77
## Coefficient H: 0.74
## Cronbach's alpha: 0.73
##
## Estimates assuming ordinal level:
##
## Ordinal Omega (total): 0.77
## Ordinal Omega (hierarch.): 0.77
## Ordinal Cronbach's alpha: 0.77
##
## Note: the normal point estimate and confidence interval for omega are based on the procedure suggested by Dunn, Baguley & Brunsden (2013) using the MBESS function ci.reliability, whereas the psych package point estimate was suggested in Revelle & Zinbarg (2008). See the help ('?scaleStructure') for more information.scaleStructure(dat=extrRev, ci=FALSE)##
## Information about this analysis:
##
## Dataframe: extrRev
## Items: all
## Observations: 2713
## Positive correlations: 10 out of 10 (100%)
##
## Estimates assuming interval level:
##
## Omega (total): 0.77
## Omega (hierarchical): 0.65
## Revelle's omega (total): 0.8
## Greatest Lower Bound (GLB): 0.8
## Coefficient H: 0.78
## Cronbach's alpha: 0.76
##
## Estimates assuming ordinal level:
##
## Ordinal Omega (total): 0.8
## Ordinal Omega (hierarch.): 0.79
## Ordinal Cronbach's alpha: 0.79
##
## Note: the normal point estimate and confidence interval for omega are based on the procedure suggested by Dunn, Baguley & Brunsden (2013) using the MBESS function ci.reliability, whereas the psych package point estimate was suggested in Revelle & Zinbarg (2008). See the help ('?scaleStructure') for more information.scaleStructure(dat=neur, ci=FALSE)##
## Information about this analysis:
##
## Dataframe: neur
## Items: all
## Observations: 2694
## Positive correlations: 10 out of 10 (100%)
##
## Estimates assuming interval level:
##
## Omega (total): 0.81
## Omega (hierarchical): 0.73
## Revelle's omega (total): 0.85
## Greatest Lower Bound (GLB): 0.85
## Coefficient H: 0.85
## Cronbach's alpha: 0.81
##
## Estimates assuming ordinal level:
##
## Ordinal Omega (total): 0.84
## Ordinal Omega (hierarch.): 0.82
## Ordinal Cronbach's alpha: 0.84
##
## Note: the normal point estimate and confidence interval for omega are based on the procedure suggested by Dunn, Baguley & Brunsden (2013) using the MBESS function ci.reliability, whereas the psych package point estimate was suggested in Revelle & Zinbarg (2008). See the help ('?scaleStructure') for more information.scaleStructure(dat=openRev, ci=FALSE)##
## Information about this analysis:
##
## Dataframe: openRev
## Items: all
## Observations: 2726
## Positive correlations: 10 out of 10 (100%)
##
## Estimates assuming interval level:
##
## Omega (total): 0.61
## Omega (hierarchical): 0.51
## Revelle's omega (total): 0.66
## Greatest Lower Bound (GLB): 0.67
## Coefficient H: 0.65
## Cronbach's alpha: 0.6
##
## Estimates assuming ordinal level:
##
## Ordinal Omega (total): 0.68
## Ordinal Omega (hierarch.): 0.68
## Ordinal Cronbach's alpha: 0.68
##
## Note: the normal point estimate and confidence interval for omega are based on the procedure suggested by Dunn, Baguley & Brunsden (2013) using the MBESS function ci.reliability, whereas the psych package point estimate was suggested in Revelle & Zinbarg (2008). See the help ('?scaleStructure') for more information.ci=FALSE indicates that we do not want the confidence interval for the estimate (although best practice suggests that this is helpful to report). The output for the agreeableness subscale from this function is as shown on the below.
Eunseong Cho and Seonghoon Kim(2020).Cronbach’s Coefficient Alpha: Well Known but Poorly Understood
Stratified Alpha
Kamata, Turhan, and Darandari’s (2003) investigation of four methods revealed that stratified-alpha was the best alternative. In Revelle and Zinbarg’s (2009) analysis of 13 formulas, McDonald’s o was recommended as the best choice. The latent class reliability coefficient (LCRC) was ranked first in an analysis by van der Ark, van der Palm, and Sijtsma (2011), which considered five techniques. Tang and Cui’s (2012) comparison of three lower bounds supported the use of Guttman’s Lambda2.
apply function without defining item strata
library(sirt)
library(psych)
bfi <- bfi
head(bfi)## A1 A2 A3 A4 A5 C1 C2 C3 C4 C5 E1 E2 E3 E4 E5 N1 N2 N3 N4 N5 O1 O2 O3 O4
## 61617 2 4 3 4 4 2 3 3 4 4 3 3 3 4 4 3 4 2 2 3 3 6 3 4
## 61618 2 4 5 2 5 5 4 4 3 4 1 1 6 4 3 3 3 3 5 5 4 2 4 3
## 61620 5 4 5 4 4 4 5 4 2 5 2 4 4 4 5 4 5 4 2 3 4 2 5 5
## 61621 4 4 6 5 5 4 4 3 5 5 5 3 4 4 4 2 5 2 4 1 3 3 4 3
## 61622 2 3 3 4 5 4 4 5 3 2 2 2 5 4 5 2 3 4 4 3 3 3 4 3
## 61623 6 6 5 6 5 6 6 6 1 3 2 1 6 5 6 3 5 2 2 3 4 3 5 6
## O5 gender education age
## 61617 3 1 NA 16
## 61618 3 2 NA 18
## 61620 2 2 NA 17
## 61621 5 2 NA 17
## 61622 3 1 NA 17
## 61623 1 2 3 21# Recoding items A1,C4,C5,E1,E2, O2,O5
bfi$A1 <- 7-bfi$A1
bfi$C4 <- 7- bfi$C4
bfi$C5 <- 7- bfi$C5
bfi$E1 <- 7- bfi$E1
bfi$E2 <- 7- bfi$E2
bfi$O2 <- 7- bfi$O2
bfi$O5 <- 7- bfi$O5
# define item strata when variables name begin with a specific letter
itemstrata <- cbind(colnames(bfi[, 1:25]), substring(colnames(bfi[, 1:25]), 1,1))
stratified.cronbach.alpha(bfi[, 1:25], itemstrata = itemstrata)## scale I alpha mean.tot var.tot alpha.stratified
## 1 total 25 0.692 104.108 152.435 0.789
## 2 A 5 0.703 23.217 20.274 NA
## 3 C 5 0.727 21.309 22.755 NA
## 4 E 5 0.762 20.723 28.113 NA
## 5 N 5 0.814 15.820 35.696 NA
## 6 O 5 0.600 22.972 16.289 NADefine subscales or item strata by user
subscale <- rep(1:5, each=5)
itemstrata <- cbind(names=colnames(bfi[, 1:25]), subscale)
itemstrata## names subscale
## [1,] "A1" "1"
## [2,] "A2" "1"
## [3,] "A3" "1"
## [4,] "A4" "1"
## [5,] "A5" "1"
## [6,] "C1" "2"
## [7,] "C2" "2"
## [8,] "C3" "2"
## [9,] "C4" "2"
## [10,] "C5" "2"
## [11,] "E1" "3"
## [12,] "E2" "3"
## [13,] "E3" "3"
## [14,] "E4" "3"
## [15,] "E5" "3"
## [16,] "N1" "4"
## [17,] "N2" "4"
## [18,] "N3" "4"
## [19,] "N4" "4"
## [20,] "N5" "4"
## [21,] "O1" "5"
## [22,] "O2" "5"
## [23,] "O3" "5"
## [24,] "O4" "5"
## [25,] "O5" "5"sirt::stratified.cronbach.alpha(bfi[, 1:25], itemstrata=itemstrata )## scale I alpha mean.tot var.tot alpha.stratified
## 1 total 25 0.692 104.108 152.435 0.789
## 2 1 5 0.703 23.217 20.274 NA
## 3 2 5 0.727 21.309 22.755 NA
## 4 3 5 0.762 20.723 28.113 NA
## 5 4 5 0.814 15.820 35.696 NA
## 6 5 5 0.600 22.972 16.289 NA