Extreme Composite Effect Presentation
Brian Syzdek
2022-06-02
Scenario- Testing with Leiter-3
| Subtest Names | Scaled Scores | Label |
|---|---|---|
| Figure Ground | 8 | Average |
| Form Completion | 8 | Average |
| Classification/Analogies | 8 | Average |
| Sequential Order | 8 | Average |
| Mean | SD |
|---|---|
| 10 | 3 |
Scenario- Testing with Leiter-3
| Subtest Names | Scaled Scores | Label |
|---|---|---|
| Figure Ground | 8 | Average |
| Form Completion | 8 | Average |
| Classification/Analogies | 8 | Average |
| Sequential Order | 8 | Average |
| Composite | Index Score | Label |
|---|---|---|
| Nonverbal IQ | 81 | Below Average |
| Mean | SD |
|---|---|
| 100 | 15 |
Scenario- Testing with Leiter-3
How can subtest scores be average, but index score is below average?
| Subtest Names | Scaled Scores | Label |
|---|---|---|
| Figure Ground | 8 | Average |
| Form Completion | 8 | Average |
| Classification/Analogies | 8 | Average |
| Sequential Order | 8 | Average |
| Composite | Index Score | Label |
|---|---|---|
| Nonverbal IQ | 81 | Below Average |
Expected Phenomenon
Explanation to Clinician
Expected Phenomenon
Explanation to Clinician
- Regression to the Mean
Expected Phenomenon
Explanation to Clinician
- Regression to the Mean
- Calculation of Index Scores
Expected Phenomenon
Explanation to Clinician
- Regression to the Mean
- Calculation of Index Scores
- Geometric Interpretation
Expected Phenomenon
Explanation to Clinician
- Regression to the Mean
- Calculation of Index Scores
- Geometric Interpretation
Explanation to Stakeholders
Regression to the Mean
- Category label = convenience and understandability
Regression to the Mean
- Category label = convenience and understandability
- Continuous variables -> categories = loss of information
| Source | Score | Interpretation |
|---|---|---|
| Subtest | 8 - 12 | Average |
| Index | 90 - 109 | Average |
Subtest scores of 8 and 12 are both “Average,” but are different (8 = “Low Average”)
Regression to the Mean
- Subtests and composites are normed separately
Regression to the Mean
- Subtests and composites are normed separately
- It is not unusual to score “Low Average” on one subtest
Regression to the Mean
- Subtests and composites are normed separately
- It is not unusual to score “Low Average” on one subtest
- It is more unusual for that pattern on several subtests : Multiple “Low Average” results -> “Below Average” overall
Calculation of Index Scores
\[ \text{standardized score} = \frac{x-\mu}{\sigma} = \frac{\text{difference from mean}}{sd} \]
Calculation of Index Scores
\[ \text{standardized score} = \frac{x-\mu}{\sigma} = \frac{\text{difference from mean}}{sd} \]
\[ \text{scaled score} = \frac{score - 10}{3} \]
Calculation of Index Scores
\[ \text{standardized score} = \frac{x-\mu}{\sigma} = \frac{\text{difference from mean}}{sd} \]
\[ \text{scaled score} = \frac{score - 10}{3} \]
\[ \frac{\text{subtest raw score } - \text{subtest mean}}{\text{subtest sd}} = \frac{\text{scaled score} - 10}{3} \]
Calculation of Index Scores
\[ \frac{\text{subtest raw score } - \text{subtest mean}}{\text{subtest sd}} = \frac{\text{scaled score} - 10}{3} \]
# Rescale scores function
standardized_score_function <- function(
raw_score,
subtest_mean,
subtest_sd,
standardized_mean,
standardized_sd)
{
(
(raw_score - subtest_mean) /
subtest_sd
) * standardized_sd + standardized_mean
}
Calculation of Index Scores
Subtest 1:
- M = 20
- SD = 5
- raw score = 10
Subtest 1
# Example to show result of different SD
standardized_score_function(
raw_score = 10,
subtest_mean = 20,
subtest_sd = 5,
standardized_mean = 10,
standardized_sd = 3
) %>%
{print(paste("Subtest 1 scaled score:", .))}
[1] "Subtest 1 scaled score: 4"
Subtest 2
- M = 20
- SD = 10
- raw score = 10
Subtest 2
standardized_score_function(
raw_score = 10,
subtest_mean = 20,
subtest_sd = 10,
standardized_mean = 10,
standardized_sd = 3
) %>%
{print(paste("Subtest 2 scaled score:", .))}
[1] "Subtest 2 scaled score: 7"
Calculation of Index Scores
Smaller relative SD -> More extreme standardized score
\[ \frac{\text{difference from mean}_\text{raw}}{ {SD}_\text{raw} \color{red} {smaller}} = \frac{\text{difference from mean}_\text{standardized}}{\text{SD}_\text{standardized}} \]
Calculation of Index Scores
\[ \frac{\text{difference from mean}_\text{raw}}{ {SD}_\text{raw} \color{red} {smaller}} = \]
\[ \frac{\text{difference from mean}_\text{standardized}}{\text{SD}_\text{standardized}} \]
Calculation of Index Scores
\[ \frac{\text{difference from mean}_\text{raw}}{ {SD}_\text{raw} \color{red} {smaller}} = \]
\[ \color{red} {Overall Increase} \]
\[ \frac{\text{difference from mean}_\text{standardized}}{\text{SD}_\text{standardized}} \]
Calculation of Index Scores
\[ \frac{\text{difference from mean}_\text{raw}}{ {SD}_\text{raw} \color{red} {smaller}} = \]
\[ \color{red} {Overall Increase} \]
\[ \frac{\text{difference from mean}_\text{standardized}}{\text{SD}_\text{standardized}} \]
This side must increase to preserve equality, but SD is fixed
Calculation of Index Scores
\[ \frac{\text{difference from mean}_\text{raw}}{ {SD}_\text{raw} \color{red} {smaller}} = \]
\[ \color{red} {Overall Increase} \]
\[ \frac{\text{difference from mean}_\text{standardized} \color {red} {Increase}}{\text{SD}_\text{standardized}} \]
This side must increase to preserve equality, but SD is fixed
Calculation of Index Scores
\[ \frac{\text{difference from mean}_\text{raw}}{ {SD}_\text{raw} \color{red} {smaller}} = \]
\[ \color{red} {Overall Increase} \]
\[ \frac{\text{difference from mean}_\text{standardized} \color {red} {Increase}}{\text{SD}_\text{standardized}} \]
This side must increase to preserve equality, but SD is fixed
\[ {SD}\text{combined} \leq {SD}\text{sum} \]
Calculation of Index Scores
Standardization of two subtests is similar to one subtest
\[ \frac{Sum(\text{subtest raw score}_\text{each} - \text{subtest mean}_\text{each})}{\text{subtest sd}_\text{combined}} \]
Calculation of Index Scores
\[ \frac{Sum(\text{subtest raw score}_\text{each} - \text{subtest mean}_\text{each})}{\text{subtest sd}_\text{combined}} \]
\[ \text{subtest sd}_\text{combined} = \sqrt{Var_{sub_1} + Var_{sub_2} + \boldsymbol{2*Cov_{sub_1/sub_2}}} \]
Calculation of Index Scores
\[ \frac{Sum(\text{subtest raw score}_\text{each} - \text{subtest mean}_\text{each})}{\text{subtest sd}_\text{combined}} \]
\[ \text{subtest sd}_\text{combined} = \sqrt{Var_{sub_1} + Var_{sub_2} + \boldsymbol{2*Cov_{sub_1/sub_2}}} \]
This combined SD can be shown be shown by combining variance of each subtest and expanding (FOIL)
\[ \frac{\sum_i((x_i - \mu_x) + (y_i - \mu_y))^2}{N} = \] \[ \frac{\sum_i((x_i - \mu_x)^2 + (y_i - \mu_y))^2 + 2(x_i - \mu_x)(y_i - \mu_y)))}{N} \\ \]
Calculation of Index Scores
Combined variance of two subtests = \[ Variance_{sub_1} + Variance_{sub_2} + \boldsymbol{2*Covariance_{sub_1/sub_2}} \]
Calculation of Index Scores
Combined variance of two subtests = \[ Variance_{sub_1} + Variance_{sub_2} + \boldsymbol{2*Covariance_{sub_1/sub_2}} \]
Covariance cannot be greater than variance of subtests
If Covariance = Variance, and all variances are equal \[ {SD}_\text{combined} = \sqrt{Var_{sub_\text{all}} + Var_{sub_\text{all}} + 2 * \color {red} {Var} _{sub_\text{all}}} \]
Calculation of Index Scores
Combined variance of two subtests = \[ Variance_{sub_1} + Variance_{sub_2} + \boldsymbol{2*Covariance_{sub_1/sub_2}} \]
Covariance cannot be greater than variance of subtests
If Covariance = Variance, and all variances are equal \[ {SD}_\text{combined} = \sqrt{Var_{sub_\text{all}} + Var_{sub_\text{all}} + 2 * \color {red} {Var} _{sub_\text{all}}} \]
\[ = \sqrt{4*\color{red}{Var}_{sub_\text{all}}} \]
Calculation of Index Scores
Combined variance of two subtests = \[ Variance_{sub_1} + Variance_{sub_2} + \boldsymbol{2*Covariance_{sub_1/sub_2}} \]
Covariance cannot be greater than variance of subtests
If Covariance = Variance, and all variances are equal \[ {SD}_\text{combined} = \sqrt{Var_{sub_\text{all}} + Var_{sub_\text{all}} + 2 * \color {red} {Var} _{sub_\text{all}}} \]
\[ = \sqrt{4*\color{red}{Var}_{sub_\text{all}}} \] \[ = 2SD \]
Calculation of Index Scores
In this same case, consider the same score difference from mean on the two subtests
\[ \frac{Sum(\text{subtest raw score}_\text{each} - \text{subtest mean}_\text{each})}{\text{subtest sd}_\text{combined}} \]
Calculation of Index Scores
One subtest
\[ \frac{\text{Difference from mean}}{\text{subtest sd}} \]
Two subtests
\[ \frac{2 * \text{Difference from mean}}{2 * \text{subtest sd}_\text{combined}} \]
Calculation of Index Scores
One subtest
\[ \frac{\text{Difference from mean}}{\text{subtest sd}} \]
Two subtests
\[ \frac{2 * \text{Difference from mean}}{2 * \text{subtest sd}_\text{combined}} \]
Equivalent …
but this is when subtests are perfectly correlated. As correlation decreases, SD decreases, and composite is more extreme.
Calculation of Index Scores
Combined variance of two subtests = \[ Variance_{sub_1} + Variance_{sub_2} + \boldsymbol{2*Covariance_{sub_1/sub_2}} \]
- Covariance cannot be greater than variance of subtests
Calculation of Index Scores
\[ \mathrm{COR}\left(X,Y\right)=\frac{\mathrm{COV} \left(X,Y\right)}{\sqrt{Var\left(X\right)\cdot V a r\left(Y\right)}} \]
- When variances equal, alternative form:
\[ {\sigma_{composite}} = \frac{\sqrt{2 + 2{\rho}}}{2}{\sigma_{pooled}} \]
Calculation of Index Scores
Appendix I: Age 7
- FC:FG correlation = .73
- FG SD = 3.3
- FC SD = 4.5
\[ .73=\frac{\mathrm{COV} \left(X,Y\right)}{\sqrt{3.3^2\cdot4.5^2}} \]
# Obtain covariance from variance and sd
covariance_func <- function(
correlation,
subtest_1_sd,
subtest_2_sd)
{
correlation *
sqrt(
subtest_1_sd**2 *
subtest_2_sd**2
)
}
FG FC covariance = 10.84
Calculation of Index Scores
Combined variance of two subtests = \[ Variance_{sub_1} + Variance_{sub_2} + \boldsymbol{2*Covariance_{sub_1/sub_2}} \]
# Combine variances using covariance, from previous function
combined_variance_func <- function(
correlation,
subtest_1_sd,
subtest_2_sd)
{
subtest_1_sd**2 +
subtest_2_sd**2 +
2 * covariance_func(correlation, subtest_1_sd, subtest_2_sd)
}
FG FC combined variance = 42.52
Calculation of Index Scores
Two subtests to scaled score:
\[ \frac{(\text{subtest raw score}_\text{each} - \text{subtest mean}_\text{each})}{\text{subtest sd}_\text{combined}} = \frac{\text{scaled score} - 10}{3} \]
\[ \frac{(8 - 10) + (8-10)}{\sqrt{42.52}} = \frac{\text{scaled score} - 10}{3} \]
Calculation of Index Scores
# Rescale composite on original subtest scale of two subtests
combined_subtest_standardized_score_func <- function(
raw_score_subtest_1,
raw_score_subtest_2,
mean_subtest_1,
mean_subtest_2,
correlation,
subtest_1_sd,
subtest_2_sd,
standardized_mean,
standardized_sd){
(
(
(raw_score_subtest_1 + raw_score_subtest_2) -
(mean_subtest_1 + mean_subtest_2)
) /
sqrt(
combined_variance_func(correlation, subtest_1_sd, subtest_2_sd)
)
) * standardized_sd + standardized_mean
}
Combined score of FG and FC
7.85
Calculation of Index Scores
# Correlations and SD's averaged across ages
all_correlations <- c(.33, .40, .39, .54, .43, .47)
all_subtest_sd <- c(3.2, 3.4, 3.6, 3.3)
all_sd_subtest_1 <- c(rep(3.2, 3), rep(3.4, 2), 3.6)
all_sd_subtest_2 <- c(3.4,3.6,3.3, 3.6, 3.3, 3.3)
# Apply the covariance function to get variance/covariance matrix
(mapply(covariance_func,
all_correlations,
all_sd_subtest_1,
all_sd_subtest_2
) %>%
# Combined variance as sum of variances and covariances
sum() *2 + sum(sapply(all_subtest_sd, function(x) x**2))) %>%
sqrt -> all_combined_variance
# -8 is sum of four subtest score difference from mean: 4(8 - 10)
(-8 / all_combined_variance) * 15 + 100 -> index_score
Consider 4 subtests with scores of 8
The NVIQ score in this case is 88
Geometric Interpretation
Cosine of angle between two vectors is equal to standardized Pearson correlation
\[ \cos\theta = \frac{a\cdot b}{\lVert{a}\rVert \, \lVert{b}\rVert} \]
Geometric Interpretation
Geometric Interpretation
Geometric Interpretation
# Same example as above to show composite calculation and plot are same
composite_example <- combined_subtest_standardized_score_func(
raw_score_subtest_1 = 12,
raw_score_subtest_2 = 12,
mean_subtest_1 = 10,
mean_subtest_2 = 10,
correlation = .73,
subtest_1_sd = 3,
subtest_2_sd = 3,
standardized_mean = 10,
standardized_sd = 3
)
The composite score using standardization formula is 12.15
Summary
Composite scores are more extreme than the average of subtest scores because:
- Regression to the mean- it is even more unlikely to have several unlikely scores
Summary
Composite scores are more extreme than the average of subtest scores because:
- Regression to the mean- it is even more unlikely to have several unlikely scores
- Index calculation- composite index variance will be (almost always) less than subtests
Summary
Composite scores are more extreme than the average of subtest scores because:
- Regression to the mean- it is even more unlikely to have several unlikely scores
- Index calculation- composite index variance will be (almost always) less than subtests
- Geometric- As correlation decreases, extremity increases
Communicate to Stakeholders
| Subtest Names | Scaled Scores | Label |
|---|---|---|
| Figure Ground | 8 | Average |
| Form Completion | 8 | Average |
| Classification/Analogies | 8 | Average |
| Sequential Order | 8 | Average |
| Composite | Index Score | Label |
|---|---|---|
| Nonverbal IQ | 81 | Below Average |
- Summarize findings
Communicate to Stakeholders
| Subtest Names | Scaled Scores | Label |
|---|---|---|
| Figure Ground | 8 | Average |
| Form Completion | 8 | Average |
| Classification/Analogies | 8 | Average |
| Sequential Order | 8 | Average |
| Composite | Index Score | Label |
|---|---|---|
| Nonverbal IQ | 81 | Below Average |
- Summarize findings
- Explain differences within categories
Communicate to Stakeholders
| Subtest Names | Scaled Scores | Label |
|---|---|---|
| Figure Ground | 8 | Average |
| Form Completion | 8 | Average |
| Classification/Analogies | 8 | Average |
| Sequential Order | 8 | Average |
| Composite | Index Score | Label |
|---|---|---|
| Nonverbal IQ | 81 | Below Average |
- Summarize findings
- Explain differences within categories
- Repeated small differences become bigger
Resources
Schneider, W. J. (2016). Why Are WJ IV Cluster Scores More Extreme Than the Average of Their Parts? A Gentle Explanation of the Composite Score Extremity Effect (Woodcock-Johnson IV Assessment Service Bulletin No. 7). Houghton Mifflin Harcourt.
Brian Syzdek: Product Manager, Stoelting Psychology | bsyzdek@stoeltingco.com