Log In

Scoring errors jeopardize tests: Poor oversight raises risk

Comments 0

Free access to myAJC for AJC subscribers.


This newspaper had reported extensively on cheating but hadn’t dug deeply into a more basic question: Just how good are the exams used to make critical decisions in schools?

No one, it turned out, had documented testing errors’ scope, causes or consequences since the 2001 No Child Left Behind Act.

Staff writer Heather Vogell decided to examine the issue during a prestigious Spencer Education Fellowship at the Columbia University Graduate School of Journalism in New York City, where she did most of the reporting.

She requested documents on testing from all 50 states and the District of Columbia, conducted scores of interviews and reviewed news stories and federal reports.

The AJC also reviewed the statistics for more than 90,000 test questions given on roughly 1,700 tests in 42 states and Washington, D.C. Vogell worked with Teachers College, Columbia University, testing expert Matthew Johnson, who is also the editor of the Journal of Educational and Behavioral Statistics.

The examination found that for nearly 9 percent of exams, one in 10 or more test items showed signs of potential problems, threatening the tests’ overall quality and raising questions about fairness. The finding was based on a statistic commonly referred to as “discrimination” (typically the “point biserial” or “item-total correlation”). Low-discrimination questions such as those the newspaper identified should often be revised or thrown out, experts say.

The statistical review and documents revealed persistent problems in the quality of questions on the standardized tests . The AJC explored those issues in a Sept. 15 story.

Today’s story is based on hundreds of pages of records on testing breakdowns from government agencies – including audits, reports, memos and correspondence between state agencies and test contractors. The newspaper scrutinized more than 100 testing failures across the country.

Most Read
Things To Do By