Surge in CRCT results raises ‘big red flag’

No other Georgia fifth grade pulled off such a feat in the past three years. It was, as one researcher put it, as extraordinary as a snowstorm in July. In Atlanta.

Atherton Principal James Berry said the scores were the product of intense tutoring.

But state education officials said last week they will investigate steep gains at Atherton and four other schools as a result of The Atlanta Journal-Constitution’s inquiries.

“It’s a big red flag,” said Kathleen Mathers, executive director of the Governor’s Office of Student Achievement. She said officials don’t know what caused the Criterion-Referenced Competency Test scores to soar, only that they require explanation.

The state has not routinely mined test data for such anomalies. But officials said it will begin to do so soon, employing widely accepted statistical methods similar to what the AJC used.

Expert: Atherton’s improvement ‘miraculous’

Atherton’s unlikely performance was one of a handful the AJC uncovered by analyzing student scores on the CRCT and retest. The surges were so far outside the norm they raise questions about whether those schools’ retest scores are valid.

As a result, the findings also suggest some schools — such as Atherton — that relied on the retest to reach academic goals might not have met federal standards.

Atherton originally placed in the 10th percentile among Georgia fifth grades on the math test, meaning 90 percent of the 1,200-plus schools scored better, the newspaper’s study shows.

After the retest, Atherton jumped to the 77th percentile. The move was unduplicated by any school statewide.

The Atherton student with what was likely the biggest gain answered just 16 math questions correctly his first time taking the test — a slightly better result than a student could expect after guessing on all 60 multiple-choice questions.

On the retest, however, the unidentified boy joined the ranks of high scorers, answering 50 questions correctly. Students needed 29 right to pass.

Two experts said the school’s rocket ride to the top tier may be too good to be true. They said educators have yet to discover methods that would cause such a jump.

“We don’t know of any interventions that do this,” said Gregory Cizek, a testing expert at the University of North Carolina.

Improving so many scores so much after 18 days of summer school, he said, “is miraculous.” He likened it to a July blizzard.

In an interview, principal Berry said the school made a tremendous summer-school effort to address failing students’ weaknesses.

“This was all but giving blood,” he said. “I pulled out every stop known to man.”

Asked whether he had any concerns about test security or score validity, Berry said: “Oh my God, I hope not. I know the people that I chose were pretty honest. I would hope that that wasn’t the case. Well, I can honestly say to you that I don’t think that that was the case at all.”

DeKalb school official suspects scoring error

DeKalb school officials are investigating because of the AJC’s questions, spokesman Dale Davis said.

Tom Bowen, vice chairman of DeKalb’s school board, said it was hard to believe that retest prep would make so much of a difference for so many students.

He suspects a score-processing error, he said, but impropriety remains a possibility. He said he welcomed the investigations.

“The children who receive these scores are not served if these scores aren’t valid,” Bowen said. “They would get a false sense of their achievement level.”

Atherton’s spectacular fifth-grade retest scores bumped the school into compliance with the federal “adequate yearly progress” standard under the No Child Left Behind Act.

This year was the first that federal authorities let Georgia districts and schools use retest scores in that calculation. It meant the retests, traditionally used to decide whether to retain children, took on new significance as a last resort for schools desperate to make the grade and avoid penalties.

In addition to undergoing closer state monitoring, schools that don’t meet federal standards lose prestige and run the risk of losing students — and the state money that comes with them — to other schools.

In the metro area, two other schools also made extraordinary gains. Like Atherton, each met federal standards only because of retest gains.

Adamsville in Atlanta had 48 percent improvement rate

In Atlanta, 19 of 19 fifth-graders at Adamsville Elementary took and passed the CRCT math retest, state data show. Statewide, only about half of retesters passed at each school. Adamsville students’ scores increased an average of 48 points, compared to the state average of 16 points.

Principal Sharon Suitt said her school, like others, identified what tasks students did poorly during the spring test and focused on them in summer school.

“It was an all-concerted effort to make sure they were successful,” she said.

Suitt conceded Adamsville’s strategy was similar to that of other schools. She said her school tried to boost student self-esteem and was “just really working to motivate students, encouraging them and letting them know they could do it.”

She said she did not have concerns about whether the exams were secure or the scores valid.

The Atlanta district has no plans to investigate, spokesman Joe Manguno said in an e-mail. The school developed a learning plan for each student, he said. Classes were taught by teachers who had demonstrated success with the spring CRCT.

“We are satisfied that the gains were valid and defensible,” Manguno wrote.

Fulton’s Parklane school shot ahead

At Fulton County’s Parklane Elementary, 22 of 24 fifth-graders who failed the math CRCT passed the retest. The school outperformed three others in Fulton that shared summer school classes at Parklane.

Principal Lee Adams said Parklane also zeroed in on students’ weaknesses. He said the summer program used a traditional teaching approach centered on basic skills. Students are more serious during the summer, he added.

Adams said he didn’t know what his school did differently from other schools to produce such large gains. But he said he, too, was not worried about test tampering.

“We’re the closest school to the district office. We make sure we’re always on our P’s and Q’s because you never know when someone is going to come out.”

Fulton officials do not see a need to investigate the gains, spokeswoman Susan Hale wrote in an e-mail. Parklane students received targeted instruction and a visit from Superintendent Cindy Loe at the beginning of the summer to encourage them.

“The summer school scores are the result of the hard work of both the teachers and students at Parklane,” she said.

Two other schools, one each in Glynn County and Gainesville, also stuck out in the newspaper’s analysis of test and retest scores. A spokesman for the Glynn district said its gains were the result of thorough remediation. A Gainesville official declined to comment.

The AJC’s study used a common statistical measurement called standard deviation to measure how much schools’ score increases differed from the average gain.

At the five schools in question, gains were more than four standard deviations above the average, the analysis showed. That meant the schools outpaced others to an unusually high degree.

“It’s concerning, and certainly it signals to you you need to look more closely to see what’s going on,” said Mathers, of the state achievement office.

Atherton’s scores were nine standard deviations above the average. “You don’t have 30 kids change on an average of nine standard deviations,” Cizek said after reviewing the newspaper’s analysis.

Experts said improprieties that could cause such gains include test tampering by an adult, student cribbing or coaching by exam monitors. Or, some teachers may have taught much more closely to the test than others — a practice some consider cheating.

State education officials will look more closely at the five schools the newspaper identified with the biggest average gains, said Stephen Pruitt, state associate superintendent for assessment and accountability. Others that were less dramatic standouts also may be examined, he said.

An investigation could include more data analysis and conversations with school officials. It also could involve what’s called an erasure analysis, which looks for an abnormal number or pattern of changed answers on students’ bubbled test sheets.

A group of state education officials will discuss the five schools at a meeting after the holidays, Pruitt said. They will begin collecting information from the schools right away and will consider each individually, he said.

The group could refer cases to the Professional Standards Commission, which polices educator credentials, or the student achievement office, which can perform a school audit. Such an audit could involve interviews with students and teachers.

Unfavorable findings could lead the state Board of Education to revoke a school’s status as meeting federal standards.

The two state education agencies have begun toughening oversight of the school-reported data used to calculate adequate yearly progress.

“We’ve got a governor and a superintendent that are very serious about the integrity of our data,” Mathers said of Gov. Sonny Perdue and Superintendent Kathy Cox.

State officials emphasized they believe the vast majority of Georgia educators are trying to play by the rules.

Some experts say high-stakes testing has fueled the motivation to cheat on standardized tests.

Tom Haladyna, a professor emeritus at Arizona State University and a testing expert who also examined the AJC’s data, noted nine schools had changes three standard deviations or higher.

“These are astounding results,” he said.

Haladyna said states should investigate unusual gains more regularly to weed out corrupted scores. In general, he and Cizek said, most teachers are honest. They have suffered over the years because of those who aren’t.

“Your school is published as a nonattaining school while these others that have maybe taken liberties are celebrated as high-achieving,” Haladyna said. “It shames all of us in education to have this happen.”