df = read.csv("clean_data.csv") # read in dataAssignment 5: Reliability & Measures Section
Reliability Analysis: Internal Consistency
Future Time Perspective Scale
Reliability Estimate
df %>%
select(FTP_1, FTP_2, FTP_3, FTP_4, FTP_5, FTP_6, FTP_7, FTP_8R, FTP_9R, FTP_10R) %>%
psych::omega(plot = FALSE)Loading required namespace: GPArotationOmega
Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
digits = digits, title = title, sl = sl, labels = labels,
plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
covar = covar)
Alpha: 0.91
G.6: 0.93
Omega Hierarchical: 0.75
Omega H asymptotic: 0.79
Omega Total 0.95
Schmid Leiman Factor loadings greater than 0.2
g F1* F2* F3* h2 h2 u2 p2 com
FTP_1 0.67 0.61 0.82 0.82 0.18 0.54 2.00
FTP_2 0.66 0.56 0.76 0.76 0.24 0.58 1.96
FTP_3 0.68 0.60 0.83 0.83 0.17 0.56 1.98
FTP_4 0.74 0.22 0.22 0.65 0.65 0.35 0.84 1.41
FTP_5 0.74 0.39 0.71 0.71 0.29 0.78 1.52
FTP_6 0.74 0.35 0.67 0.67 0.33 0.82 1.43
FTP_7 0.73 0.23 0.63 0.63 0.37 0.84 1.39
FTP_8R 0.50 0.60 0.61 0.61 0.39 0.41 1.96
FTP_9R 0.54 0.24 0.48 0.58 0.58 0.42 0.50 2.39
FTP_10R 0.49 0.63 0.65 0.65 0.35 0.36 2.00
With Sums of squares of:
g F1* F2* F3* h2
4.31 1.20 0.38 1.02 4.85
general/max 0.89 max/min = 12.62
mean percent general = 0.62 with sd = 0.18 and cv of 0.29
Explained Common Variance of the general factor = 0.62
The degrees of freedom are 18 and the fit is 0.17
The number of observations was 274 with Chi Square = 46.65 with prob < 0.00024
The root mean square of the residuals is 0.02
The df corrected root mean square of the residuals is 0.03
RMSEA index = 0.076 and the 90 % confidence intervals are 0.05 0.104
BIC = -54.38
Compare this with the adequacy of just a general factor and no group factors
The degrees of freedom for just the general factor are 35 and the fit is 1.88
The number of observations was 274 with Chi Square = 504.85 with prob < 1.3e-84
The root mean square of the residuals is 0.15
The df corrected root mean square of the residuals is 0.17
RMSEA index = 0.221 and the 90 % confidence intervals are 0.205 0.239
BIC = 308.39
Measures of factor score adequacy
g F1* F2* F3*
Correlation of scores with factors 0.89 0.81 0.53 0.81
Multiple R square of scores with factors 0.78 0.65 0.28 0.65
Minimum correlation of factor score estimates 0.57 0.31 -0.44 0.30
Total, General and Subset omega for each subset
g F1* F2* F3*
Omega total for total scores and subscales 0.95 0.92 0.86 0.81
Omega general for total scores and subscales 0.75 0.52 0.74 0.36
Omega group for total scores and subscales 0.13 0.40 0.12 0.45Descriptive Summary
df %>%
summarise(mean(FTP_mean, na.rm = T),
sd(FTP_mean, na.rm = T)) mean(FTP_mean, na.rm = T) sd(FTP_mean, na.rm = T)
1 4.09558 1.355519Appreciation of Time Scale
Reliability Estimate
df %>%
select(ART_1, ART_2, ART_3, ART_4) %>%
psych::omega(plot = FALSE)Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
The estimated weights for the factor scores are probably incorrect. Try a
different factor score estimation method.Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
ultra-Heywood case was detected. Examine the results carefullyWarning in cov2cor(t(w) %*% r %*% w): diag(V) had non-positive or NA entries;
the non-finite result may be dubiousOmega
Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip,
digits = digits, title = title, sl = sl, labels = labels,
plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option,
covar = covar)
Alpha: 0.82
G.6: 0.79
Omega Hierarchical: 0.76
Omega H asymptotic: 0.87
Omega Total 0.87
Schmid Leiman Factor loadings greater than 0.2
g F1* F2* F3* h2 h2 u2 p2 com
ART_1 0.81 0.70 0.70 0.30 0.94 1.06
ART_2 0.64 0.47 0.22 0.64 0.64 0.36 0.64 2.10
ART_3 0.54 0.53 0.62 0.62 0.38 0.46 2.12
ART_4 0.81 -0.21 0.68 0.68 0.32 0.97 1.15
With Sums of squares of:
g F1* F2* F3* h2
2.02 0.00 0.51 0.13 1.76
general/max 1.15 max/min = Inf
mean percent general = 0.75 with sd = 0.25 and cv of 0.33
Explained Common Variance of the general factor = 0.76
The degrees of freedom are -3 and the fit is 0
The number of observations was 274 with Chi Square = 0 with prob < NA
The root mean square of the residuals is 0
The df corrected root mean square of the residuals is NA
Compare this with the adequacy of just a general factor and no group factors
The degrees of freedom for just the general factor are 2 and the fit is 0.14
The number of observations was 274 with Chi Square = 37.12 with prob < 8.7e-09
The root mean square of the residuals is 0.1
The df corrected root mean square of the residuals is 0.17
RMSEA index = 0.253 and the 90 % confidence intervals are 0.186 0.328
BIC = 25.89
Measures of factor score adequacy
g F1* F2* F3*
Correlation of scores with factors 0.91 0 0.71 0.53
Multiple R square of scores with factors 0.83 0 0.51 0.29
Minimum correlation of factor score estimates 0.66 -1 0.01 -0.43
Total, General and Subset omega for each subset
g F1* F2* F3*
Omega total for total scores and subscales 0.87 NA 0.77 0.82
Omega general for total scores and subscales 0.76 NA 0.45 0.82
Omega group for total scores and subscales 0.10 NA 0.33 0.00Descriptive Summary
df %>%
summarise(mean(ART_mean, na.rm = T),
sd(ART_mean, na.rm = T)) mean(ART_mean, na.rm = T) sd(ART_mean, na.rm = T)
1 5.326642 1.165971Validity Analysis: Divergent Validity
df %>% cor.test( ~ FTP_mean + ART_mean, data = .)
Pearson's product-moment correlation
data: FTP_mean and ART_mean
t = 4.0624, df = 272, p-value = 6.36e-05
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
0.1241888 0.3478115
sample estimates:
cor
0.239169 fa_data = df %>%
select(FTP_1, FTP_2, FTP_3, FTP_4, FTP_5, FTP_6, FTP_7, FTP_8R, FTP_9R, FTP_10R, ART_1, ART_2, ART_3, ART_4) %>%
mutate(across(everything(), ~ .x - mean(.x, na.rm = T)))
M1 = fa(fa_data,
nfactors = 2, # specifcy that I want to estimate 2 factors
fm ="pa", # use principle axes factoring method
rotate = "promax", # allow factors to be correlated
scores="regression") # get factor scores by a factor score regression method
fa.diagram(M1, cex = 0.7,
rsize = 0.5,
e.size = 0.1) # SEM diagram of factor loadings
M1 # df of loadings and fit statisticsFactor Analysis using method = pa
Call: fa(r = fa_data, nfactors = 2, rotate = "promax", scores = "regression",
fm = "pa")
Standardized loadings (pattern matrix) based upon correlation matrix
PA1 PA2 h2 u2 com
FTP_1 0.74 0.20 0.65 0.35 1.1
FTP_2 0.72 0.17 0.61 0.39 1.1
FTP_3 0.72 0.27 0.68 0.32 1.3
FTP_4 0.82 -0.08 0.65 0.35 1.0
FTP_5 0.70 0.03 0.51 0.49 1.0
FTP_6 0.70 0.12 0.55 0.45 1.1
FTP_7 0.80 -0.03 0.63 0.37 1.0
FTP_8R 0.63 -0.13 0.37 0.63 1.1
FTP_9R 0.67 -0.03 0.44 0.56 1.0
FTP_10R 0.64 -0.33 0.41 0.59 1.5
ART_1 -0.13 0.79 0.59 0.41 1.1
ART_2 0.22 0.68 0.59 0.41 1.2
ART_3 0.01 0.59 0.35 0.65 1.0
ART_4 -0.05 0.80 0.62 0.38 1.0
PA1 PA2
SS loadings 5.26 2.41
Proportion Var 0.38 0.17
Cumulative Var 0.38 0.55
Proportion Explained 0.69 0.31
Cumulative Proportion 0.69 1.00
With factor correlations of
PA1 PA2
PA1 1.00 0.24
PA2 0.24 1.00
Mean item complexity = 1.1
Test of the hypothesis that 2 factors are sufficient.
df null model = 91 with the objective function = 8.83 with Chi Square = 2361.87
df of the model are 64 and the objective function was 1.75
The root mean square of the residuals (RMSR) is 0.07
The df corrected root mean square of the residuals is 0.08
The harmonic n.obs is 274 with the empirical chi square 249.65 with prob < 9.6e-24
The total n.obs was 274 with Likelihood Chi Square = 466.74 with prob < 1.6e-62
Tucker Lewis Index of factoring reliability = 0.747
RMSEA index = 0.152 and the 90 % confidence intervals are 0.139 0.165
BIC = 107.5
Fit based upon off diagonal values = 0.97
Measures of factor score adequacy
PA1 PA2
Correlation of (regression) scores with factors 0.96 0.92
Multiple R square of scores with factors 0.93 0.85
Minimum correlation of possible factor scores 0.85 0.70plot.psych(M1) # plot loadings
Measures Section Purpose
The Measures section of a paper describes the instruments that were used to operationalize the constructs of interest in a study. This section establishes the reliability of the scale, meaning how consistently the scale measures a given construct, as well as the validity of the measures, meaning how appropriate the measures are for the inferences for which we want to use them.
Measures Section Prose
Future Time Perspective
Participant’s perceptions of their future were measured using the Future Time Perspective Scale (Carstensen & Lang, 1996), a 10-item measure in which individuals rated the truthfulness of a series of statements about themselves (e.g., “I could do anything I want in the future”) on a seven-point scale (1 = Very Untrue, 7 = Very True). Three items are reverse coded and all items are averaged to create a single score in which larger values correspond to more expansive views of future time (M = 4.10, SD = 1.36). Using McDonald’s omega (McDonald, 1999), I determined that the items demonstrated strong internal consistency (\(\omega_t\) = 0.95) given a standard threshold of \(\omega_t\) > 0.80 to indicate suitable internal consistency.
Appreciation of Remaining Time Scale
The value participants placed on their future time was measured with the Appreciation of Remaining Time Scale (Carstensen et al., 2024), a four-item measure in which individuals respond to statements such as, “I savor the good times and know the bad times pass,” by rating how true such statements are about themselves on a 1 (Very Untrue) to 7 (Very True) scale. Items are averaged to create a single score in which larger values reflect greater appreciation of remaining time (M = 5.33, SD = 1.17). McDonald’s omega (McDonald, 1999), using a threshold of \(\omega_t\) > 0.80 for suitable internal consistency, indicated that this scale is reliable (\(\omega_t\) = 0.87).
Validity of Measures
The moderately low correlation between FTP and ART (r = .24) as well as the exploratory factor analysis indicating that the items of each scale load onto separate factors (see above), indicates divergent validity for these items. Thus, I would conclude that these measure tap distinct constructs pertaining to cognitive and affective appraisals of future time.
References
Carstensen, L. L., Chu, L., Matteson, T. J., & Growney, C. M. (2024). What’s time got to do with it? Appreciation of time influences social goals and emotional well-being. Psychology and Aging, 39(8), 833–853. https://doi.org/10.1037/pag0000856
Carstensen, L. L., & Lang, F. R. (1996). Future time perspective scale. Unpublished manuscript, Stanford University.
McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum.