Comparison of Assessment Texts vs. Instructional Texts in Two Programs

NOTE: These results are PRELIMINARY. Please DO NOT CITE or DISTRIBUTE without author’s permission.

A series of graphs in the tabs below shows overall-text complexity as well as word-level factors that contribute to text complexity for levels A to P texts in two popular guided-reading programs: A to Z and LLI.

Each dot in a graph represents a text, which is color-coded either yellow (indicating assessment/benchmark texts) or blue (indicating instructional texts). Passage title, its guided-reading level, text-characteristic score, and genre pop up if a cursor is hovered over a dot.

Lines indicate trajectories of level-specific average scores (either means or medians) across A to P levels. These lines are expected to increase as the guided reading levels go up because higher levels of texts are expected to be more complex and to include more words with challenging characteristics (e.g., longer, more abstract, learned at later years, and rarer). Also expected is little discrepancy between the lines within and across the two programs because passages placed at a same level, on average, should have similar characteristics across the two passage-types (assessment vs. instructional) as well as across the two programs.

Failing these would have serious consequences as it would indicate a guided-reading level does not have a consistent meaning across the text-types or the programs.

Overall Complexity

Graphs below show overall text complexity scores from the Early-Reader Lexile Analyzer.

Findings

  • A-Z’s assessment texts at levels C through G are strikingly lower in complexity as compared to instructional texts at these levels. This is evident with the A-Z assessment texts’ level-specific mean line is tracing the least complex instructional passage within each levels from C to G. The same pattern is observed at level K for the A-Z program.
  • In contrast, a few level K assessment texts within the A-Z program are considerably more complex than other level K passages. In fact, these passages have the same score (~=750L) as the most complex passages at levels O and P. This is pushing the mean complexity score up for the assessment texts at level K than that for the instructional text, which is evident in a bump with the yellow line above the blue line.
  • Compared to the A-Z program, LLI program’s assessment and instructional passages, on average, are similar in complexity within levels, as evident in less gap between the two mean lines. Exceptions are levels F, H, and N.
  • In both programs, instructional passages’ complexity varies considerably within a level.
  • Complexity varies quite a bit with A-Z’s assessment passages leveled at and above J.
  • Mean and Median lines show fairly similar patterns. One exception is that with medians, the difference between A-Z program’s assessment vs. instructional texts at level O is more pronounced (see a larger dip with the yellow line in the second graph.)

Decoding-d

Graphs below show the distributions of decoding demand scores (in percentiles) from the Early-Reader Lexile Analyzer. The first graph shows level-specific mean lines while the second graph shows median lines. Both graphs show similar patterns.

Findings

  • Decoding demand scores are all over the map. It is shocking that some of the most demanding texts in the A-Z program are at Levels A, B, and E.
  • In both programs, there is considerable variability in decoding demand among instructional texts, especially at lower levels (for K and G1.1).
  • Mean/median lines show a zig zag pattern, indicating that decoding demand does not increase from one leve to next. In some cases, there is a huge drop in average decoding demand, for example, from level F to level G, among A-Z’s assessment texts.
  • Within certain levels (e.g., A, C, F, G in the A-Z program), instructional passages and assessment passages differ considerably in decoding demand, as indicated by the vertical gap between the yellow and blue lines.

Syllable_ct

Graphs below show the distributions of scores for the cout of syllables in words across reading levels A through P. Raw scores in the first graph vary from 0 (few words with many syllables) to 8 (more words with more syllables), while scores in the second graph are percentiles.

Findings

  • In the A-Z program, some of the lower level instructional passages have the highest syllable-count scores.
  • The syllable-count scores fluctuate from one leve to next, as shown in zig-zag lines. Large ups and downs are evident especially with the assesment passages in both programs (see for example, the yellow line for the A-Z program at levels F, G, and H.)
  • At certain levels (e.g., at A-G levels in the A-Z program), instructional passages and assessment passages, on average, differ considerably in decoding demand, as indicated by the vertical gap between the yellow and blue lines.

AoA

A graph below shows the distributions of age of acquisition scores across A-P reading levels. Possible raw scores range from 1 to 25, with lower score indicating more words in the text are known by younger readers.

Findings

Abstract

Graphs below show distributions of word abstractness scores across A-P reading levels. Possible raw scores range from 1 to 25 (see the first graph), with lower score indicating more words in the text are known by younger readers.

Findings

Rareness

A graoh below shows the distributions of word rareness scores across A-P reading levels. Possible scores range from 0 (less rare, easier) to 6 (more rare, more difficult). It is derived as an inverse of word frequency from MetaMetrics’ 1.39 billion-words corpus with 93,000 K-16 texts, normalized to link to the frequencies in Carroll, Davies, and Richman’s frequency 5 million word list.

Key findings

Count_Passages

Table 1. Number of Texts by Level
dataset level title
A-Z A 14
A-Z B 14
A-Z C 14
A-Z D 14
A-Z E 14
A-Z F 14
A-Z G 14
A-Z H 14
A-Z I 14
A-Z J 14
A-Z K 16
A-Z L 16
A-Z M 16
A-Z N 16
A-Z O 16
A-Z P 16
LLI A 14
LLI B 14
LLI C 14
LLI D 14
LLI E 14
LLI F 14
LLI G 14
LLI H 14
LLI I 14
LLI J 14
LLI K 14
LLI L 16
LLI M 16
LLI N 16
LLI O 14
LLI P 14