Overreaction to NAEP scores does little to help students
For many educators, it’s panic time again, following the latest results of the National Assessment of Educational Progress, aka the “Nation’s Report Card,” as it has been labeled by the people who are behind it. They are not shy about trumpeting the quality of their own work when they say that it is the gold standard assessment for measuring U.S. students’ knowledge and skills in math, reading, writing, science, U.S. history and civics.
Who knows what the fate of this self-proclaimed gold standard will be, given that it relies on data collected by the Common Core of Data for its sampling? CCD has been decapitated by the blitzkrieg from Elon Musk’s chain saw. But don’t worry. It might be back in a week.
When NAEP scores are viewed as a calamity, they produce predictable responses. Critics’ solutions tend to be the same ones advanced before the test results were announced: more charter schools, vouchers for private school education, ending diversity, equity and inclusion programs and so on from one end of the spectrum; and school meal programs, more school nurses, safe and filth-free school buildings and more from the other.
Instructionally, the same issues that have driven the Reading Wars are offered to solve the latest crisis: more phonics, less balanced instruction from one side; and less phonics, more balanced instruction from the other.
NAEP scores have actually remained fairly stable for decades, with modest gains, until COVID-19 and the new era of social media addiction changed the game. A downward fluctuation in test scores will typically trigger alarm bells in the headlines and raise the specter of an illiterate nation riddled by crime, economic instability and stagnation. We’re reminded of the CEO of an ice company whose board wondered why ice sales declined during cold spells, because profits should always increase, no matter what the circumstances are.
We find NAEP scores neither alarming nor insignificant. Yet, we doubt that we are facing the epic crisis trumpeted by the media and politicians. We’ve noted several problems with the development, administration and announcement of NAEP scores, including:
- NAEP claims to measure “reading achievement” as if it were a single, widely agreed-upon, clearly known, readily measured (i.e., testable) entity. Reading success is multifaceted and varies by occasion, as you might know from reading a T.S. Elliot poem and The Atlanta Journal-Constitution. It is like other measures of success. Do the written exams and driving tests required for a driver’s license mean you are a competent driver in rush hour traffic? Does taking the hourlong NAEP exam mean you are or aren’t a good reader? Bear in mind that people are usually highly motivated to pass their driving exams. The NAEP test is given to students who accept the honor of being randomly selected in their school to take an extra test. We wonder how many students take this honor seriously. Might testing exhaustion and apathy be a factor in any nationally administered test results?
- NAEP’s creation of achievement levels is problematic, controversial and misleading. In 1992, NAEP created cut scores for achievement levels: Advanced, Proficient, Basic and Below Basic. That decision was made against the recommendations of testing specialists who continue to express concerns. The levels are inherently subjective and ultimately decided by political appointees, and have produced confusion about the distinction between “Basic” and “Proficient.” Despite NAEP’s explicit cautions, some try to equate “proficient” with “reading at grade level” — itself a constructed measure based on test score standard deviations — conflating two amorphous concepts.
- NAEP’s national scores and their headlines are misleading. Here’s a typical headline: “A Dismal Report Card on Math and Reading.” Yet, the decline behind the headline obscures the fact that scores dropped meaningfully in only five states between 2022 and 2024. The overall drop since COVID was driven by an increase in students scoring below Basic, who are mostly from economically disadvantaged demographic groups. Students’ higher score levels were essentially unchanged.
- NAEP uses the weakest standard for determining statistically meaningful changes in scores from year to year. The ideal standard in education research is less than 1 chance in 1,000 that a difference is directly attributable to something other than random variation or measurement error. A less reliable standard is less than 1 in 100, and the minimal acceptable standard is less than 5 in 100. NAEP uses that minimal standard, which becomes especially questionable given that large national samples increase the odds of reaching the ideal, higher standard.
- NAEP scores, for more than 30 years, have varied relatively little. NAEP uses a 500-point scale to report scores. Since 1992, the national average scores at fourth grade range from a low of 214 (in 1993) to a high of 223 (in 2015), mostly on an upward trend. Even the post-COVID dips in the past two cycles are higher than the lowest point in the past.
NAEP says little about causes of increases or decreases in scores, let alone about what should be done to improve matters. That doesn’t prevent people from interpreting them in line with their confirmation bias. Those who want to promote their preferred reforms consistently cite NAEP to suggest that schools are doing an awful job of teaching reading and math.
Worries about NAEP scores also have produced wealth for publishers, consultants, grant recipients and others who profit from “disaster capitalism,” a term originally applied to profiteering following a natural disaster but also appropriate to constructed problems like NAEP test scores. What isn’t clear is how edupreneurs benefit kids in learning The three R’s and enjoying their benefits.
David Reinking is a retired professor at Clemson and the University of Georgia, an inductee in the Reading Hall of Fame, and former co-editor of Reading Research Quarterly and Journal of Literacy Research. Peter Smagorinsky is a retired professor at UGA, an inductee in the Reading Hall of Fame, and former co-editor of Research in the Teaching of English.
