October 10, 2023 Quarto

Steve Hoffman

NAEP Report Card: Mathematics

https://www.nationsreportcard.gov/mathematics/?grade=8#

In 2022, the National Assessment of Educational Progress (NAEP) mathematics assessment was administered to representative samples of fourth- and eighth-grade students in the nation, states, the District of Columbia, Puerto Rico, Department of Defense schools, and 26 participating large urban districts. The mathematics assessment at grade 12 was last administered nationally in 2019. The assessments measured students’ knowledge and skills in mathematics and their ability to solve problems in mathematical and real-world contexts.

Let’s look at 8th grade

In 2022, twenty-six percent of eighth-grade students performed at or above the NAEP Proficient level on the mathematics assessment, which was 7 percentage points lower compared to 2019 and was 11 percentage points higher than the percentage in 1990, the first assessment year. Sixty-two percent of eighth-grade students performed at or above the NAEP Basic level in mathematics, which was 7 percentage points lower than in 2019 and 10 percentage points higher than in 1990.

Setup Analysis

## LOAD PACKAGES
library(tidyverse)
library(readxl)

## READ IN DATA ####
Gr8 <- read_excel("NAEP_Gr8_Math.xlsx")

# Make tibble
Gr8 <- as_tibble(Gr8)

# Pivot longer, first select
Gr8_long <- select(Gr8, Year, Avg_All, Avg_Asian, Avg_Black, Avg_Hispanic, Avg_White)

# Pivot
Gr8_long <- Gr8_long |>
  pivot_longer(!Year,
    names_to = "Race_Ethnicity",
    values_to = "Average"
  )

For this analysis only…

Focus on data post NCLB
“Main” NAEP data from 2003 through 2022
School-reported race/ethnicity (pre-2011 guidelines)
- Asian/Pacific Islander, Black, Hispanic, White
- For this analysis, no American Indian/Alaska Native analysis nor Two or more races analysis
- Accommodations Permitted

Standards-based reporting on NAEP

Percent above proficient
Percent above basic
% above a standard changes differently for different groups (race, poverty, etc.)
reporting % passing biases comparisons of trends

Before Pandemic

Gains in NAEP average 8th grade math 2003 - 2019

Group	2003	2019	gain
Asian	291	310	19
Black	252	260	8
Hispanic	259	268	9
White	288	292	4
All	278	282	4

Before Pandemic (standards based)

Gains in NAEP percent proficient or advanced (2003 - 2019)

Group	Increase %Basic	Increase %Proficient
Asian	8	19
Black	8	7
Hispanic	9	8
White	0	7
All	1	5

Loss in NAEP average 8th grade math 2019 - 2022

Group	2019	2022	loss
Asian	310	304	6
Black	260	253	7
Hispanic	268	261	7
White	292	285	7
All	282	274	8

Loss in NAEP percent proficient or advanced (2019 - 2022)

Group	Change %Basic	Change %Proficient
Asian	-2	-6
Black	-9	-5
Hispanic	-8	-6
White	-6	-9
All	-7	-8

Trends in average Math8 scale scores

# Create Scale Plot
Math8plot <- 
ggplot(Gr8_long, aes(x = Year, y = Average)) +
  geom_line(mapping = aes(color = Race_Ethnicity)) +
  theme_classic() +
  labs(x = "Year",
       y = "NAEP Scale Scores",
       title = "Main NAEP National Average Math Score Grade 8",
       subtitle = "2003 - 2022",
       color = "Legend") + 
  scale_x_continuous(limits = c(2003, 2022)) +
  scale_y_continuous(limits = c(240, 340)) +
  geom_hline(yintercept = 262, linetype = "dotted") +
  geom_hline(yintercept = 299, linetype = "dotted") +
  geom_hline(yintercept = 333, linetype = "dotted") +
  scale_color_manual(values = c("Avg_Asian" = "blue", "Avg_White" = "hot pink", "Avg_All" = "red", "Avg_Hispanic" =  "brown", "Avg_Black" = "Black")) +
  annotate(
    geom = "label", x = 2004, y = 337,
    label = "Advanced"
  ) + 
  annotate(
    geom = "label", x = 2003.9, y = 303,
    label = "Proficient"
  ) + 
  annotate(
    geom = "label", x = 2003.5, y = 266,
    label = "Basic"
  )

Scale Score Plot

Growth in SD since 2003

# Calculate from 2003
Gr8 <- Gr8 |>
  mutate(Diff_All = Avg_All - 278) |>
  mutate(Diff_Asian = Avg_Asian - 278) |>
  mutate(Diff_Black = Avg_Black - 278) |>
  mutate(Diff_Hispanic = Avg_Hispanic - 278) |>
  mutate(Diff_White = Avg_White - 278) 

# Add column calculating difference in SD since 2003
Gr8 <- Gr8 |>
  mutate(Diff_sd_All = Diff_All/SD_All) |>
  mutate(Diff_sd_Asian = Diff_Asian/SD_All) |>  
  mutate(Diff_sd_Black = Diff_Black/SD_All) |>
  mutate(Diff_sd_Hispanic = Diff_Hispanic/SD_All) |>
  mutate(Diff_sd_White = Diff_White/SD_All)

Calculate change in SD by Race

# Calculate change 2022 - 2003 
Gr8$Diff_sd_All[1] - Gr8$Diff_sd_All[10]

[1] -0.1025641

Gr8$Diff_sd_Asian[1] - Gr8$Diff_sd_Asian[10]

[1] 0.3055556

Gr8$Diff_sd_Black[1] - Gr8$Diff_sd_Black[10]

[1] 0.08119658

Gr8$Diff_sd_Hispanic[1] - Gr8$Diff_sd_Hispanic[10]

[1] 0.09188034

Gr8$Diff_sd_White[1] - Gr8$Diff_sd_White[10]

[1] -0.0982906

Plot the difference in SD

# Create plot
SDMath8.plot <- ggplot(Gr8) +
  geom_line(aes(Year, Diff_sd_Asian), color = "blue") +
  geom_line(aes(Year, Diff_sd_White), color = "hot pink") +
  geom_line(aes(Year, Diff_sd_All), color = "red") +
  geom_line(aes(Year, Diff_sd_Hispanic), color = "brown") +
  geom_line(aes(Year, Diff_sd_Black), color = "black") +
  labs(x = "Year",
       y = "Difference in Standard Deviation") +
  labs(title = "Main NAEP National Average Math Score Grade 8", 
       subtitle = "Difference in average scores (in SD) since 2003") +
  scale_x_continuous(limits = c(2003, 2028)) +
  scale_y_continuous(limits = c(-1, 1)) +
  annotate(
    geom = "label", x = 2025.5, y = 0.667,
    label = "Asian average up 0.31"
  ) +
  annotate(
    geom = "label", x = 2025.5, y = 0.179,
    label = "White average down 0.10"
  ) + 
  annotate(
    geom = "label", x = 2025.5, y = -0.103,
    label = "Overall average down 0.10"
  ) +
  annotate(
    geom = "label", x = 2025.5, y = -0.436,
    label = "Hispanic average up 0.09"
  ) +
  annotate(
    geom = "label", x = 2025.5, y = -0.641,
    label = "Black average up 0.08"
  )

Change in SD by Race (2003 - 2022)

Asian/White Gap

Looking at 8th grade math on NAEP:

Did “the gap” grow between 2003 & 2019?
Scale scores
“effect size” (standard deviation units)
% Basic
% Proficient
Percentile rank

Scale scores

Gains in NAEP average 8th grade math 2003 - 2019

Year	Asian	White	Difference
2003	291	288	3
2019	310	292	18
Gain	19	4	15

A/W Gap in average scale scores grew by 15 points from 2003 to 2019

Compare to all students

In 2003, mean = 278 & sd = 36 In 2019, mean = 282 & sd = 40

Year	Asian	White	Difference
2003	0.36	0.28	0.08
2019	0.70	0.25	0.45
Difference	0.34	-0.03	0.37

Percent Basic

Gap analysis of 8th grade NAEP math

Percent at or above basic

Year	Asian	White	Difference
2003	78%	80%	-2
2019	86%	80%	6

A/W Gap in % above basic grew by 8 percentage points from 2003 to 2019

Percent Proficient

Gap analysis of 8th grade NAEP math

Percent at or above proficient

Year	Asian	White	Difference
2003	43%	37%	6
2019	62%	44%	18

A/W Gap in % above proficient grew by 12 percentage points from 2003 to 2019

% Proficient post pandemic

Percent at or above proficient

Year	Asian	White	Difference
2019	62%	44%	18
2022	56%	35%	21

A/W Gap in % above proficient grew by 3 percentage points from 2019 to 2022

Average Percentile Rank

# percentiles, 2003 & 2019 & 2022
pnorm(291, mean=278, sd=36) # Asian 2003

[1] 0.6409918

pnorm(288, mean=278, sd=36) # White 2003

[1] 0.6094085

pnorm(310, mean=282, sd=40) # Asian 2019

[1] 0.7580363

pnorm(292, mean=282, sd=40) # White 2019

[1] 0.5987063

pnorm(304, mean=274, sd=39) # Asian 2019

[1] 0.7791218

pnorm(285, mean=274, sd=39) # White 2019

[1] 0.6110479

Average Percentile Rank

Year	Asian	White	Difference
2003	64%	61%	3%
2019	76%	60%	16%
2022	78%	61%	17%

A/W Gap in percentile rank grew by 13 percentile points from 2003 to 2019

A/W Gap in percentile rank grew by 1 percentile point from 2019 to 2022