Cheating Our Children: The story so far
Using a computer analysis to examine test scores at Atlanta Public Schools, The Atlanta Journal-Constitution found ample reason to suspect widespread cheating on standardized tests. After breaking the APS cheating scandal, the AJC embarked on an analysis of test scores from every public school district in the United States. How the story unfolded:
2008
Staff writer Heather Vogell and database specialist John Perry analyze scores on the Criterion Referenced Competency Test, or CRCT, and find what appear to be unusual score changes at one APS school and three other metro schools. They publish a story and begin to get phone calls from APS teachers and others alleging extensive cheating at APS. Superintendent Beverly Hall, a nationally known educator and innovator who built her reputation on improved scores at APS, dismisses the possibility of widespread cheating on her watch.
2009
Perry and Vogell develop a computer analysis that flags schools with unusual changes in test scores. Perry calculates the probability that a class’s improvement from one year to the next could have occurred by chance. In the case of some schools, the odds were greater than a billion to one against. Because of the AJC’s reporting, the Governor’s Office of Student Achievement decides to carry out an erasure analysis on all CRCT answer sheets statewide.
2010
The state analysis flags 58 Atlanta schools for excessive erasures in which wrong answers were changed to right answers. The state orders APS to investigate. Staff writer Alan Judd joins the AJC’s investigation, and he and Vogell report that part of the gain in both test scores and graduation rates claimed by Beverly Hall are illusions. In May, while waiting for a flight in Denver, Hall sends an email to a former board member saying, “I received a very prestigious award. The rest of the country are not buying into the AJC representation of widespread cheating.” In the summer, after months of investigation the school district’s specially appointed commission finds that widespread cheating was limited to 12 schools. Gov. Sonny Perdue rejects the finding as “woefully inadequate in both scope and depth” and orders up his own investigation. He appoints a former attorney general and a former district attorney to lead the inquiry and puts dozens of GBI agents at their disposal. Late this year, Hall announces that she will retire the following summer.
2011
The state investigators gather evidence, interviewing scores of APS teachers, administrators and staff. In July, Perdue’s successor, Gov. Nathan Deal, releases the results – an 800-page report saying that close to 200 educators cheated on the CRCT at 44 schools. Errol Davis, the former chancellor of the University System of Georgia, takes over as superintendent with a vow that no one who cheated on tests will be allowed in front of Atlanta schoolchildren again. In November, as APS pursues the process of firing teachers and administrators suspected of cheating, Perry, Vogell, Judd and staff writer M.B. Pell take the AJC’s cheating analysis nationwide. They acquire multiple years’ worth of standardized test scores for every public school in America and begin running the numbers.
2012
In the spring, the team reports that 196 districts around the country exhibit suspicious patterns of test scores that, in Atlanta, indicated cheating. Later the analysis shows that dozens of U.S. schools awarded the nation’s highest educational honor – being named a National Blue Ribbon School – also had highly unlikely test scores. Today and next Sunday, the AJC focuses on lax oversight of standardized testing across the nation.
In the spring of 2008, students at a Mobile, Ala., middle school discovered something weird on their yearly achievement tests: someone, somehow, had changed their answers. But when their teacher alerted the principal, he suggested she proceed with caution.
“Sleep on it,” the principal said.
When the teacher reported the apparent cheating to Alabama’s state education department, it ordered a thorough investigation of Scarborough Middle School. But not too thorough. Computer analyses that might detect organized tampering, the state superintendent of education says, would have amounted to a “witch hunt.”
MORE IN THE SERIES: Part one: Suspicious school test scores across the nation | Part two: Suspect scores at premier schools | Part three: Help on tests can cross the line | Part Five: States can't ensure test integrity
The Mobile episode, detailed in interviews and public records, illustrates the haphazard manner in which many states and school districts handle reports of cheating on high-stakes achievement tests, The Atlanta Journal-Constitution has found. Officials often minimize such allegations, treating them as mere aberrations: one-time occurrences best dealt with in isolation.
This is the case in several cities the Journal-Constitution identified earlier this year as having, along with Atlanta, extreme concentrations of suspicious test scores: Mobile, Dallas, Houston, Detroit, Baltimore, St. Louis, and East St. Louis, Ill. The newspaper recently examined more than 130 cheating cases from those cities, a review that provides a ground-level view of cheating and its consequences.
In some cases, investigations uncovered wrongdoing and led to punishment for a handful of educators. In others, inquiries glossed over glaring irregularities. Nearly always, officials focused narrowly on a single classroom or, at most, a single school — the approach the Atlanta Public Schools used for years before a scandal over systemic cheating erupted three years ago.
It is as if the Centers for Disease Control and Prevention handled food-poisoning outbreaks one victim at a time, without trying to find the source or scope of contamination. Patterns of misconduct go unnoticed, and cheating flourishes.
The cases reviewed by the Journal-Constitution do not represent all the cheating allegations in the seven districts, only those for which local and state education agencies would release files. Nevertheless, they reveal broad suspicions, often confirmed, of cheating in many districts highlighted by the newspaper’s earlier analysis, which found nearly 200 districts with highly unlikely swings in test scores. They also depict a subculture of dishonesty in a noble profession: teachers who brazenly provide answers to their students, administrators who convene groups that erase and correct students’ responses, school district officials who look the other way when wrongdoing emerges.
In East St. Louis, cheating was “accepted practice” at Annette Officer Elementary, the school district said in a recent report. Test scores rocketed and plunged over several years, a telltale sign of tampering. But it wasn’t until this spring, after a teacher reported improprieties, that the district opened an inquiry.
The teacher said that before testing began, an administrator instructed her to tell students to mark lightly on their answer sheets. That way, the administrator said, erasures would not “look like someone else had made them.”
The school’s principal tested one class himself, according to records of a district investigation, and repeatedly ruled out answers until students saw that only the correct response remained.
“I didn’t give you any answers, did I?” the principal asked after the test.
“No,” the students answered.
“Good,” he said, “because I would get in trouble for giving you answers.”
The cheating scheme was so elaborate, officials alleged, that an administrator devised a code to warn the school staff if outsiders showed up during testing. As a team of officials looking into testing violations approached the school last March, they heard the administrator broadcast the alert on the public address system: “Will Abraham Lincoln please come to the office?”
The principal denied wrongdoing, but he and two instructional coaches quit, bringing the investigation to a close.
The district saw no need to check other East St. Louis schools for irregularities, Beth Shepperd, an assistant superintendent, said last week.
“This was public in our community,” Shepperd said. “If any teacher at another school felt there was a concern, that teacher had an opportunity to come forward.”
Fewer than a dozen teachers or administrators lost their jobs over cheating in the 130 cases the Journal-Constitution examined. Three of them, all from the same middle school in Houston, also faced criminal charges. The majority of cases, including those in which investigators said they found “severe” or “sloppy” or “unallowable” violations, ended with no more than a warning, if even that.
Texas officials, for instance, determined that in 2008 a Houston teacher instructed her fifth-grade students to “fix” specific answers she identified as incorrect on their state math tests. The Texas Education Agency called this a “serious violation of testing procedures,” but admonished the teacher only that the episode “may be used to enhance any subsequent sanction” if she broke the rules again.
Investigating allegations of cheating remains a low priority in many states, despite high-profile scandals in Atlanta, Philadelphia, the District of Columbia and other school districts. Just 10 states budget for such inquiries, according to a recent survey of state education agencies by the Journal-Constitution. Those states set aside from $5,000 to $250,000 a year for investigations; Georgia spent at least $2.2 million on cheating cases in Atlanta and Dougherty County. In many states, officials acknowledge they can afford rigorous inquiries only in the most blatant cases.
The depth of an inquiry “depends to some degree on what information we get, if it’s isolated or systemic, the amount of proof,” said Jan Ellis, a spokeswoman for the Michigan Department of Education, echoing comments from officials across the nation.
“There’s an extensive effort to make sure we have the best investigative techniques available,” Ellis said.
In Mobile, the largest school district in Alabama, Superintendent Martha Peek acknowledges that some teachers or administrators may cheat, and she says she reviews test scores for clues, such as gains or decreases that defy the odds. Data lines the walls of her conference room: laminated color charts detailing the number of schools meeting national standards, the percentage of students who pass state achievement tests, the graduation rates of the district’s high schools, and on and on. At the head of one row of charts, Peek posted a cautionary quotation from Winston Churchill: “No matter how beautiful the strategy, you should occasionally look at the results.”
“Good, bad, or whatever, we want true scores,” Peek said. “I’d rather have bad scores than have scores that are not legitimate. We can fix academic problems. You cannot fix problems with integrity.”
But Peek said she was unaware that, at roughly the same time as the cheating case unfolded at Scarborough Middle, close to a dozen other Mobile schools posted scores with inexplicable gains and decreases.
One of them was Booker T. Washington Middle School, where in 2008 sixth-graders scored in the 8th percentile statewide in reading, lower than 92 percent of all Alabama students who took the same test. A year later, when they were seventh-graders, these same students scored in the 81st percentile. Then, in the eighth grade, their rank plummeted to the 3rd percentile.
At Spencer-Westlawn Elementary, entire grades of students elevated their state rankings by as much as 67, 73, even 86 percentage points in a single year. Earlier this month, the U.S. Department of Education rewarded these gains with its highest honor: it named Spencer-Westlawn a National Blue Ribbon School.
Peek said the fluctuations reflected “instability of the student population or the instructional leader or the school staff.”
Many testing experts discount such factors, though. They say teaching methods, turnover in school staff and student mobility have only minimal effects on test scores.
‘We’re going to let the situation rest’
With school out for summer, Scarborough Middle School, eight miles northwest of downtown Mobile, was a grim sight.
Dust blew off a barren lawn in front of the drab, brown-brick building. Rows of portable classrooms sat atop concrete blocks, their windows covered with plywood to ward off squatters. A sign announcing a renovation project gamely promised “Progress through Education.”
Progress has long eluded Scarborough. With almost 90 percent of its students living in poverty, it has persistently ranked among the lowest-scoring schools in one of the lowest-performing states in America. When Mobile enabled students to transfer from failing schools last year, more left Scarborough than any other.
In 2008, with test scores lagging far behind district and state averages, someone set out to improve Scarborough’s fortunes — illicitly, if necessary.
The scheme came to light after teacher Mandy Phillips passed out the answer sheets for the second portion of her eighth-graders’ math tests.
Three students immediately said their work from a prior testing day had been erased and changed. Simultaneously, a student in another classroom told her teacher the same thing.
Phillips first notified a testing coordinator, according to files at the Alabama State Department of Education. But the coordinator merely “shrugged her shoulders,” records say, so Phillips went to the principal, Charles L. Smith. His response: “Sleep on it” while he decided “which way I need to go with this.”
Two days later, according to the state files, Smith told Phillips that “someone higher up” in the school district had advised him, “We’re going to let the situation rest, and we need to keep quiet.”
District officials dispute Smith’s claim that he consulted a supervisor about the matter. Neither Smith nor his lawyer, J. Malcolm Jackson III, responded to numerous messages requesting an interview. In court records, Jackson said, “There was no evidence presented that Smith engaged or assisted in the irregularities.”
Before Phillips’ second conversation with Smith, according to state records, the school’s achievement specialist had retested the four students whose answer sheets had been altered. But, students later told investigators, the specialist gave them the correct answers.
“Don’t tell,” she was reported to have said. “I don’t want the school to get in trouble.”
‘We assume everything is going as it should’
If no one told, no Alabama school would get in trouble for cheating.
The state analyzes test results from individual schools or districts only “if there is a reason to question [their] integrity,” said Gloria Turner, the director of student assessment for Alabama’s education department.
“We don’t just go looking,” Turner said. “We assume everything is going as it should.”
Alabama’s superintendent of education, Tommy Bice, dismissed the value of systematic screening methods such as erasure analysis, in which computers scan for suspicious changes to answers.
“You start doing that, you’re on a witch hunt,” Bice said. “It’s a demoralizing thing unless there’s a reason. We try to be a state where we deal with the people who make bad choices individually and not set policy based on that.”
Twenty-six states analyze erasures or other changes that correct students’ answers, according to the Journal-Constitution’s survey of education agencies. At least seven states decided to begin or resume the analyses following Atlanta’s cheating scandal, in which the state found a virtually impossible number of wrong-to-right erasures.
Such analyses alone do not necessarily detect, or deter, cheaters.
Texas has for years paid its testing contractor to check answer sheets for excessive erasures. But the state does not use that analysis to ferret out suspicious patterns of erasures in specific schools or districts.
“It has typically been used as supportive evidence, not something that launches an investigation,” said Debbie Ratcliffe, a spokeswoman for Texas’ education department.
“It’s not illegal to erase on tests,” Ratcliffe said. “There can be legitimate reasons for that.”
Most major cheating cases in Texas have come to light through whistleblowers or conscientious district officials.
A Dallas inquiry began when a school district statistician noticed in 2009 that students at Lang Middle School had suddenly excelled on state tests. Students who failed the math test as sixth- and seventh-graders not only exceeded state standards in the eighth grade, some posted perfect scores. A computer program identified large numbers of Lang students with “highly unlikely” increases, according to a state report, “not the type of incremental improvement normally associated with improved scores based on hard work.”
Only then did the state turn to its erasure analysis. It showed that almost 6 percent of responses had been changed on Lang’s answer sheets; the statewide average was less than 1/10 of 1 percent. Of the erasures at Lang, 98 percent turned wrong answers into correct ones.
Although irregularities pervaded Lang’s testing, the state focused on the eighth-grade math exam, which students had to pass to advance to high school.
Officials threw out the scores of 429 students and ordered them to take the test again during the summer. Eighty percent had passed in the spring, with 63 percent scoring at the advanced level. In the summer, under closer scrutiny, 44 percent passed, 4 percent with advanced scores.
Investigators identified more than a dozen people who had access to the test papers but never determined who orchestrated the cheating. The state punished two people who had already left the Dallas schools: the principal, whose teaching license was placed on probation for a year for failing to adequately supervise the school’s testing, and the testing coordinator, whose license was suspended one year for failing to secure answer sheets. Both denied wrongdoing.
A degree of ambiguity persisted.
“We don’t even know who it was” who cheated, the Dallas school superintendent at the time, Michael Hinojosa, told reporters in 2009. But he added: “People that were most responsible for this are no longer with the district.”
Hinojosa became the superintendent in Cobb County, Georgia’s second-largest school district, in 2011. In an interview, he said the Lang case demonstrated Dallas’ vigilance in detecting irregularities.
“We couldn’t actually get to the bottom line,” he said. “But we did everything that was in our authority to do.”
The investigation went no further.
Principal ‘seemed appropriately dismayed’
In January 2009, an anonymous teacher tipped off Michigan officials to alleged cheating at Detroit’s Bethune Academy, an inner-city school that has ranked among the worst 5 percent in the state. Bethune had met federal standards, known as adequate yearly progress, just three times in nine years, but the rate of students passing state tests had soared — doubled, in some grades — the previous year.
The teacher alleged that the school’s principal “directs the staff to have the students write the answers on a piece of paper instead of their answer documents. Then the central office collects all the answer documents and pieces of paper and goes into the office and changes the answers.”
The state held the complaint until the next time the Detroit schools administered achievement tests — nine months later.
Two state officials opened the investigation with an unannounced visit to Bethune. Their first stop: the principal’s office.
The principal “seemed appropriately dismayed” over the allegations, the officials wrote later, and she “immediately denied there was any wrongdoing.”
Next, they spoke to a special education teacher implicated in the anonymous complaint. She “noted nothing she felt was an irregularity in testing,” the officials wrote. Another teacher named in the complaint had “limited time” to speak, they said, but “reported no apparent irregularities.”
One of the officials visited the room where the school stored test materials and thumbed through a stack of papers from each of about 20 boxes. From this visual inspection, he concluded that answer sheets had a “normal” number of erasures.
“These were actually kid documents,” said Paul Stemmer, a state testing official.
Finally, the officials interviewed four students and two more teachers, all in the presence of an administrator from the school system’s headquarters. The students “noted nothing unusual or different” about the exams, and the teachers “noted no concerns.”
“Nor,” the officials wrote, “did they know of any other teachers who might have concerns.”
They concluded: “There appears to be no material evidence of the allegations.”
The investigation included no erasure analysis, no comparison of students’ scores to those from previous years, no collection of any empirical data at all.
A spokeswoman for the Detroit school district, Jennifer Mrozowski, suggested the allegation warranted no investigation.
“It’s unfortunate the principal, state officials and school staff had to spend valuable time … pursuing what amounted to an anonymous tip by someone who did not witness any wrongdoing and was simply ‘suspicious,’ ” Mrozowski wrote in an email. “More follow-up, such as erasure analysis and other time-consuming and resource-draining investigations, would have been a waste of time.”
State officials, however, are paying closer attention to unusual patterns in test scores and soon will begin notifying school districts of suspicious findings, Stemmer said. But at the time of the Bethune investigation, he said, the state could have taken no action without “incontrovertible evidence that something happened.”
‘He clearly did not choose to do his job’
When Mobile teacher Mandy Phillips reported cheating at Scarborough to Alabama’s education agency, officials there immediately ordered an investigation — by the Mobile school system.
The state agency was “eager for an investigation to begin to determine the truth of the reports and a determination of responsibility,” a Mobile administrator wrote in a memo. “The state department has serious concerns for the entire 8th grade and then wonders about 6th and 7th grade test validity as well.”
“Immediately,” Peek, the superintendent, said recently, “everything went into action. They did a thorough investigation.”
The district assigned the investigation to a school resource officer, whose summary of the case reads like a police report on a routine crime. One student said 10 to 15 of her answers had been changed; another said he counted 28 erasures. A third teacher said her students had also reported tampering.
But the report indicates the officer questioned only eighth-grade teachers and a few of their students, disregarding the concerns about other grades.
No one analyzed test scores for the entire eighth grade, much less for other classes.
The investigation placed blame for the episode on the testing coordinator, Karen Williams, for failing to report a security breach; the achievement specialist, Linda Ruffin, for violating rules when she re-tested students; and the principal, Smith, for failing to report testing irregularities.
Williams did not respond to a request for an interview. In a statement issued through her lawyer, Ruffin denied cheating and said she “never willfully or intentionally violated any testing security policies.” Both women settled cases with the state by accepting brief suspensions of their teaching certificates.
Authorities reserved the toughest punishment for Smith: they revoked his educator’s license, rendering him unemployable in any public school.
“He clearly did not choose to do his job,” Larry Craven, general counsel of Alabama’s education department, said in an interview.
A revocation is permanent, Craven said, although educators may later apply for reinstatement.
“There are people who are given a second chance,” he said. Most are young educators who, several years after a violation, can demonstrate they’ve matured and learned from their mistakes. Restoring a revoked certificate, he said, is “very, very rare.”
Which makes what happened to Smith all the more puzzling. After Smith appealed, state officials returned his license. Mobile then re-hired him, as an assistant principal at another middle school.
Williams and Ruffin still work for the school district, as well — at Scarborough.
But, said Peek, the superintendent, “They don’t have any more contact with testing.”
For Alabama’s state education agency, which had ordered a complete investigation, the disciplinary actions concluded the Scarborough case. The episode, state officials said in recent interviews, raised no questions about broader test-security problems. As a one-time violation, they said, it did not suggest a need to systematically screen for signs of cheating statewide or even in the rest of Mobile.
“Absent compelling evidence to suggest we have widespread cheating, we don’t think that would be a good use of our limited funds,” said Turner, the state’s assessment director.
“But we do take cheating seriously.”
-- Staff writers M.B. Pell and Wayne Washington contributed to this article.
About the Author
