Lessons learned

The AJC used freedom of information laws to collect test scores from 49 states — 14,743 school districts — to look for the sort of patterns that suggested cheating.

Among the discoveries by AJC staffers Alan Judd, Heather Vogell, John Perry, M.B. Pell and Dayton Daily News database specialist Ken McCall:

● Improbable scores were twice as likely to appear in charter schools. Under their contracts, charters make promises to deliver specific student scores, and their survival and funding depend on reaching those targets.

● The newspaper found changes in test scores that were statistically improbable in nearly 20 cities, with swings in scores that were virtually impossible in about a half-dozen. Tampering was cited as the most likely explanation. In some cities, the AJC found so many dramatic shifts in scores that the odds of that happening by chance are one in 10 billion.

● In some cities, the results for entire grades of students jumped two, three or more times the amount expected in one year. The next year, when children moved to a new grade, their scores tanked. (Please note that the AJC did look at student mobility year to year and whether the composition of the class had changed dramatically.)

● Though high-poverty city schools were more likely to have questionable scores, the analysis found improbable scores also showed up in an exclusive public school for the gifted on the Upper West Side of Manhattan. And they appeared in a rural district roughly 70 miles south of Chicago with one school.

Along with our own database reporters, the AJC consulted outside experts to assess our analysis, including University of Georgia statistics professor Jaxk Reeves, director of the Statistical Consulting Center.

To be clear, the new AJC national analysis doesn’t establish beyond any doubt that cheating occurred. But it points to the same troubling pattern later verified in Atlanta schools to be test-tampering after a probe by an outraged Gov. Sonny Perdue.

Led by two former top prosecutors, the Perdue investigation entailed 2,100 interviews and 800,000 documents and led to more than 80 confessions of cheating.

State investigators accused 38 principals of participating in test-tampering. Cheating was confirmed in 44 of 56 schools examined.

The Perdue probe proved that the sharp spikes in student achievement followed by meteoric dips were not natural occurrences as system leaders maintained, but a seismograph of shame.

The Perdue report toppled the much-celebrated regime of Beverly Hall and led to extensive upheaval in the leadership of Atlanta schools.

The findings also underscored what is a universal truth: Hold people accountable to standards, benchmarks or quotas that they feel are unrelenting, unrealistic and unfair and some will cheat.

“We are putting way too much pressure on people to raise scores at a very large clip without holding them accountable for how they are doing it,” Daniel Koretz, a Harvard Graduate School of Education testing expert, told the AJC.

That’s an important point now that many states, including Georgia, are adopting teacher-evaluation systems in which student scores play a significant role. We have to ask: By basing evaluations on student testing, are we using too narrow a lens to judge in our schools and overlooking positives that are not reflected in a single score?

Most of the districts with troubling test score swings were rural and urban districts steeped in poverty. The AJC investigation ought to serve as a catalyst for a national debate over whether schools teaching the least-advantaged and most-challenging students are being held to unattainable standards and whether test scores are a fair way to judge success.

That debate has to begin and end with these questions: Who is honestly succeeding with poor students and how are they doing it?