The stain of cheating spread unchecked across 44 Atlanta schools before the state finally stepped in and cleaned it up. But across the country, oversight remains so haphazard that most states cannot guarantee the integrity of their standardized tests, The Atlanta Journal-Constitution has found.
Poor oversight means that cheating scandals in other states are inevitable. It also undermines a national education policy built on test scores, which the states and local districts use to fire teachers, close schools and direct millions of dollars in funding.
The AJC’s survey of the 50 state education departments found that many states do not use basic test security measures designed to stop cheating on tests. And most states make almost no attempt to screen test results for irregularities.
For example, a computer analysis of erasures on test papers and a statistical analysis of improbable gains on tests are both proven ways to catch cheating. Georgia uses one of those methods, but Alabama uses neither.
The survey reveals other wildly inconsistent practices around the country: Some states require outside investigations of cheating in school districts, but most states permit districts to investigate themselves; some states look for unusual increases in test scores, but most don’t; about half send out independent monitors to oversee testing, about half do not.
At the root of the inconsistencies: The United States has a clear national strategy of administering make-or-break tests to nearly every schoolchild, but it has no national strategy to ensure the tests’ integrity.
Ten years after the No Child Left Behind Act made testing the keystone of U.S. education policy, the federal government has yet to issue standards, guidelines or even recommendations on how to prevent and detect cheating.
For years, federal officials have ignored warnings from government watchdog agencies and testing experts that inadequate security can lead to system cheating on a scale that undermines the entire system of testing. Systemic cheating gives education officials a false understanding of student achievement and, more important, robs struggling students of access to the extra help they need to improve.
Cheating scandals have surfaced in several major cities, and a Journal-Constitution analysis of test scores earlier this year suggests a nationwide problem. The newspaper reported in March that 196 school districts exhibited patterns of suspicious test scores similar to those in Atlanta, where the patterns proved to be cheating.
And if cheating is widespread, it is likely to get worse. As more states and districts tie teacher evaluations, bonuses and pay to test results, the motivation to cheat will only increase in the years to come.
For now, the nation spends as much as $760 million a year on testing required by NCLB, but it spends little to ensure the test results are legitimate.
“To spend all this money and all this energy on testing, and the one area where we haven’t devoted the same energy is standardizing the administration of the test to deter cheating,” said Wayne Camara, vice president of research at The College Board, which administers the SAT and Advanced Placement tests. “To have better or standardized procedures would limit opportunities for cheating.”
Congress seized a measure of state and local authority over education when it passed No Child Left Behind and created the requirement for high-stakes tests. But federal officials say any directive from them to ensure that the test results are valid would be an overreach.
Camara and other testing experts say the federal government should take a more active role in helping states identify and stop cheating.
“Without any kind of standardization, states are left to make it up on their own or borrow from their friends,” said Scott Norton, the former assessment director for the Louisiana Department of Education, said. “I’m not proposing a set of rules that every state must follow, but I think a set of best practices would be helpful.”
The U.S. Education Department will not do even that.
Secretary of Education Arne Duncan, in a recent interview with the AJC, said test security is best left to state and local officials.
“So much of this is best done with thoughtful leaders at the state level, not a new complicated federal bureaucracy,” Duncan said. “I don’t think anyone wants a national testing police.”
The secretary also told the AJC he sees little incentive for educators to cheat.
“As anyone can see from the testing scandal in Atlanta, cheating was not worth it,” Duncan said. “That was a disaster for the city and a major black mark. Other places have a tremendous incentive not to cheat and become embroiled in something like that.”
Good test scores make everybody happy
Poor testing oversight is often the result of education officials ignoring what no one wants to see.
In Atlanta, for example, accusations of cheating were met with denials not only from district administrators, principals and teachers, but from local business leaders.
“There is typically little to no incentive for anyone to take threats to test security seriously,” said Greg Cizek, a professor and testing expert at the University of North Carolina at Chapel Hill. “Educators are happy when test scores go up; parents are happy when their children do well; students are pleased when they are declared to be ‘proficient’; the public is assuaged when all schools appear to be increasing learning.”
He said that aggressively pursuing test security measures is a “lose-lose situation in most cases.”
With no motivation from the federal government and with little reason to look for cheating on their own, most states designed security systems that have failed to find blatant cheating even when the evidence is right in front of them.
Records show some state officials failed to act when whistleblowers stepped forward. Some states did nothing to investigate schools where students posted almost impossible gains on tests from one year to the next.
Part of the problem is a lack of looking. Of the 46 states that responded to the AJC survey:
- 24 states, Georgia among them, did not conduct an analysis looking for improbable test improvements in 2012. A statistical analysis of test results can take many forms, but all examine test results for increases in scores that are so extraordinary that they are unlikely to result from teaching alone.
- 21 did not look for an improbable number of changes from wrong to right answers in 2012. Some states, including Georgia, have their testing contractors feed answer sheets through a scanner that can detect erasures and evaluate whether a wrong answer was erased and replaced by a right one. A certain percentage of such changes can signal cheating.
- Thirty-four states did not screen for unusually similar answers that would suggest that students copied from one another or that a teacher filled in answers on multiple tests.
The numbers are even more grim than they appear.
Some states that conduct screening hope that just the threat of getting caught is enough to discourage educators from cheating. These states, like Texas, perform erasure analyses but don’t actually use the results to initiate investigations. Texas doesn’t even share the results with its school districts.
The Pennsylvania Department of Education paid its testing contractor to conduct an erasure analysis in 2009. Even though the results suggested cheating in Philadelphia, Pittsburg, Scranton and several other districts, the department did not investigate until last year after the Philadelphia Public School Notebook, a nonprofit news organization, exposed the analysis.
Even when states do use screening results to spur investigations, state officials say they often accept explanations for the score increases like a new teacher or a new curriculum. Testing experts say these are unlikely explanations for radical score increases.
Thirty-seven states allow school districts to conduct cheating investigations of their own employees, according to the AJC survey. Testing experts say an independent agency should always investigate signs of cheating, not district officials, who benefit from increased test scores.
Many states make it easier for educators to cheat by using weak security procedures.
Forty-one allow teachers to proctor tests for their own students. The proctor’s responsibilities include ensuring the security of the test; for example, making sure no unauthorized materials are used, enforcing time limits and reporting irregularities.
State and district records from multiple states show that some teachers are not above guiding their students to the right answers.
A teacher at a Phoenix elementary school, for example, told a colleague that she’d used red and green M&Ms during a test to nudge students toward the right answers. If she set a red M&M on a child’s desk, that was a signal that the pupil had the wrong answer and should stop and do the problem over again. If she put a green one on the desk, that meant the child selected the right answer and should move on.
The threat that a state monitor may look in on a test can curb this kind of cheating, according to experts. Yet only 25 states used independent monitors in 2012, and most of them deployed fewer than 20 monitors to cover hundreds or thousands of school. One state used only a single monitor.
These weak security systems allow administrators to shuffle unprepared students from grade to grade, while congratulating themselves for raising test scores.
Georgia officials could have stopped this kind of abuse in Atlanta years before state officials put an end to it, according to records obtained by the AJC.
In July 2005, four Atlanta high school teachers who said they knew of widespread cheating on graduation tests offered to meet with the Georgia Department of Education to discuss the problem, according to a copy of the letter their attorney wrote to the department.
“I am reluctant to report the complaints to the APS superintendent because these four teachers have informed me that they fear retribution,” said the letter, obtained by the AJC through an open records request.
By way of response, state officials did not offer to meet with the teachers. Instead, they simply informed the school district of the complaint – standard procedure for the state when it received a report of cheating, the officials said in a response letter. Nothing ever came of the teachers’ complaint.
Cheating in Atlanta schools continued unabated for four more years.
Raise the stakes, raise the security
When Congress passed No Child Left Behind in late 2001, lawmakers placed tremendous pressure on educators to improve student achievement. They did not, however, pressure the states to ensure the improvements were genuine.
That’s because when the law was written, no one understood the consequences of weak safeguards, said U.S. Sen. Johnny Isakson, R-Ga, who, as a member of the U.S. House, helped draft No Child Left Behind.
“I am the first to say that when debating No Child Left Behind, nobody contemplated a situation like what happened in Atlanta,” Isakson said.
Some experts assert, however, that cheating was a predictable response to NCLB.
“If you think you can have high stakes without high security and still have trustworthy measures, you’re kidding yourself,” said Scott Marion, associate director at the Center for Assessment, a not-for-profit consulting firm that works with about 30 states to develop tests and teacher accountability systems.
Marion says tests are still largely trustworthy, but cheating is a threat to the validity of scores.
Sharon Rideau, an elementary and middle school teacher in Phoenix, wrote her doctoral thesis on cheating. Rideau’s survey of more than 3,000 Arizona teachers in 2008 revealed 50 percent either had cheated themselves or knew a colleague who cheated.
“I think it happens much, much more now,” Rideau said. “It happened before NCLB and now we have all this pressure on us; it’s had a great impact.”
In addition to the Atlanta scandal, widespread cheating also broke out in Baltimore and El Paso, Texas. Currently, cheating investigations are under way in Philadelphia, Pittsburgh and other districts in Pennsylvania, in Toledo and Columbus, Ohio, and in St. Louis.
Missouri’s state auditor launched an investigation of St. Louis Public Schools after the AJC published its analysis finding unusual score patterns in the district. The analysis flagged more than one in six classes in St. Louis in some years.
Last year, the district received an allegation that educators at Herzog Elementary School in St. Louis were told to look at the test questions and then prep their students on them, according to records detailing the allegation. Fifth-grade students were told to redo portions of the test, and one student taking a makeup said the answers were already filled in.
In 2011, 43 percent of Herzog students scored proficient on the state math test. In 2012, after the school faced greater scrutiny and a change in some staff, only 8 percent of students scored proficient.
Ohio Auditor of State cq Dave Yost launched an investigation earlier this year to determine whether systemic cheating at three Ohio school districts was the result of lax oversight by the state’s Education Department. The results are expected early next year.
But if the states have weak test security, they are only guilty of following the federal government’s lead.
In 2009, the U.S. Government Accountability Office issued a report warning of “gaps” in state security policies. The agency concluded that unless the Department of Education began placing “proper emphasis on this important issue, some states may continue to rely on inadequate test security procedures that could affect the [tests’] reliability and validity.”
The GAO noted that testing companies and state education leaders were working together to publish a book on best practices for testing. When the guidelines came out in 2010, the agency said, the U.S. Department of Education should use them to examine states’ security protocols.
But the 130-page handbook contains only five pages on test security and mentions cheating in one subsection. The “best practices” are described only in vague terms.
“Procedures should be in place to prevent potential breaches of security,” the book says.
The chapter did take care to outline several ways that states should protect testing companies’ intellectual property.
Regardless federal education officials had little interest in following GAO recommendations.
“We believe that our current practices are sufficient to ensure that appropriate test security policies and procedures are promulgated and implemented at the state level,” Thelma Melendez de Santa Ana, the assistant secretary of education, told the GAO in a letter. The department already did some monitoring of test security practices and any additional security measures were “best handled locally based on consideration of risk and cost factors.”
Isakson said the Department of Education should develop test security best practices, but the states should decide whether to put them in place.
“Yes, it’s right to have best practices,” Isakson said. “The first question you’re going to hear from the states though is, ‘That’s great, but are you going to pay for it?’”
Cost does pose a problem, according to the AJC survey.
Of the states that never conducted any screening for cheating, 60 percent cited the expense.
“The Iowa Department of Education has no staff or appropriation to handle state-level test security measures,” wrote Staci Hupp, the Iowa Department of Education communications director, in the AJC survey. “If the Legislature feels it is important for Iowa schools to have tighter test security measures, we will implement them.”
But the costs of looking for cheating are minimal compared to the price tag of administering the tests. Massachusetts, a state with strong security procedures, paid $36,000 to conduct test screening and $140,000 to conduct investigations in 2012, according to the survey. That’s 0.7 percent of the $27 million the state spent to administer tests.
Are states willing to give up flexibility?
It’s not that federal officials don’t understand the risks posed by weak security systems, said Camara, the research vice president at the College Board.
He believes they decline to develop a standard set of best practices for political reasons.
After challenging state and local control over education by requiring the tests, federal officials didn’t want to start another fight over test security, Camara said. Wresting control of education from local school districts is a good way to make enemies of state legislators, district superintendents, school board members and parents.
“The states must mandate some standard security controls so we can be confident in the scores,” Camara said. “Some states are politically averse to this.”
The U.S. Department of Education does require states to provide evidence that they have testing manuals and documents that address security and the reporting of testing irregularities.
But states can do just enough to say they have a security policy, like requiring teachers to sign a pledge saying they won’t cheat or requiring teachers to undergo training on proper testing procedures.
Atlanta Public Schools required educators to sign pledges not to cheat. That didn’t prevent the nation’s worst cheating scandal.
U.S. Department of Education officials said they could not recall ever rejecting a state’s security policy.
When teachers’ pay is tied to test scores
Beginning in February the federal government began issuing waivers for No Child Left Behind that lift the requirement that nearly all children demonstrate proficiency in math and reading by 2014. They also enable states to design new school accountability standards that de-emphasize testing and introduce other ways to measure success. So far 33 states, including Georgia, have been granted the waiver.
But the era of high-stakes testing is not over. States are still required to administer tests. In fact, many states and school districts are developing plans to tie teacher pay and evaluations to student test scores.
Chicago teachers left classes earlier this month to protest, among other issues, a proposal to evaluate teachers based in part on student test performance.
“If you think there’s cheating now under school accountability, wait until what you see under teacher accountability,” said Scott Marion, the testing consultant.
The future of testing is being directed to a large degree by two organizations made up of states. The Partnership for Assessment of Readiness for College and Careers and the Smarter Balanced Consortium are each working to develop a common approach to testing for their members based on an agreed set of education standards approved by 45 states.
The Smarter Balanced Consortium has not yet discussed how testing data will be used to identify irregularities or what kind of security measures will be employed, but it plans to take up that issue in the coming months, said Tony Alpert, the chief operating officer at the Smarter Balanced Consortium.
Getting all the states to commit to strong security standards may be tough.
Three of the consortium’s member states said they don’t look for cheating because they have seen little evidence cheating takes place in their schools.
In one of those states, Nevada, the number of cheating allegations has increased in recent years, and department officials acknowledge that it’s a problem. State Superintendent James Guthrie has said he wants to explore screening methods for cheating.
But Richard Vineyard, the state’s assistant assessment director, said he does not understand why his boss wants to do that.
“It might make sense for us to look for cheating, but if we had money to put into our assessment program, that probably wouldn’t be our first priority,” Vineyard said.
The U.S. Department of Education did hold a symposium in February on testing integrity. It plans to release a summary of the discussions, perhaps this fall, but the summary will not contain a description of best practices, officials said.
The summary will note what experts have said and then advise the states and school districts that they may use the information as they see fit.
Staff writers Heather Vogell and Alan Judd contributed to this article. Amy Harris interviewed Education Secretary Arne Duncan in West Virginia on the AJC’s behalf.
About the Author
