ATL superintendent loses shine

The new image that Hall created, however, often runs counter to reality.

School officials have repeatedly distorted or misrepresented key statistics, fashioning outsize impressions of student achievement and, at the same time, an exaggerated depiction of Hall as savior of a long-troubled system, an investigation by The Atlanta Journal-Constitution shows.

Even many of her critics acknowledge Hall has improved the city’s schools. They complain, though, that she too often punctuates her administration’s accomplishments with an exclamation point where a period would suffice.

In fact, the school district’s problems run so deep that Gov. Sonny Perdue took the rare step Wednesday of ordering a special investigation to look into allegations of widespread cheating and, perhaps, other irregularities.

Hall insisted during an interview Thursday that Atlanta’s gains during her tenure are real. “We have stayed the course for a very long time in a very difficult environment,” she said, referring to the district’s mostly poor, highly transient student body. In the interview with AJC editors and reporters, Hall also asserted that no cheating has been proved, any cheating was perpetrated by unscrupulous individuals, and that the aggressive teacher accountability system that she initiated had nothing to do with any wrongdoing.

“It’s always possible,” she said, “for some human being to cheat.”

But the district’s skewed data call into question the two feats most responsible for Hall’s national stature: the dramatic increase in the graduation rate and the skyrocketing scores on state and national achievement tests.

The graduation rate leapt almost 30 percentage points between 2003 and 2009. But that surge, the AJC reported a week ago, likely occurred because of a mass exodus of students from Atlanta high schools during the three school years that ended in 2005. During that time, the district removed from its rolls 30 percent of all pupils in grades nine through 12 — roughly 16,000 students.

Almost 7,000 students, the district said last week, were “no shows” who didn’t reappear at the beginning of a school year. Those students were counted as transfers, even though officials had no idea whether they actually had dropped out of school or, if they hadn’t, where they had gone. The district said it was following state guidelines for classifying missing students.

Regardless, with so many students off the books, calculating a higher graduation rate became much easier.

Like the graduation rate, Atlanta’s scores on the Criterion-Referenced Competency Test — the CRCT, the state’s primary measure of student performance — are in dispute.

In 2009, an AJC analysis found statistically improbable increases on that year’s CRCT at some Atlanta schools. A state examination of test papers later detected thousands of erasures that improved students’ scores in two-thirds of the district’s elementary and middle schools. On the 2010 test, given under tight security, scores at most of those schools sank, by an average of 11 percent. In the schools not suspected of cheating, failure rates decreased by 1 percentage point.

The district may have taken other steps to manipulate CRCT scores at some schools, the AJC found.

For a month before schools administer the test each spring, records show, transfers spike at Forrest Hill Academy, an alternative school for students with academic or behavioral problems. Concentrated at Forrest Hill, contained by wrought-iron and chain-link fences, these poor-performing students no longer are liabilities to their home schools. Instead, their CRCT scores count against Forrest Hill, where failure rates on portions of the exam run as high as 97 percent.

As the CRCT scandal deepened this spring and summer, Hall increasingly pointed to Atlanta’s results on another standardized test: the National Assessment of Educational Progress, or NAEP. Atlanta students, Hall wrote in a district newsletter, “demonstrated the largest gains” not only among urban districts, but also in comparison to “schools within large cities and the nation overall.”

To Hall, the NAEP results validate the district’s accomplishments, and her own.

But some of the gains there, too, are illusory, the newspaper found. Atlanta still lags behind most other big-city districts, as well as Georgia and the nation at large, and it has not reduced the achievement gap between black and white students to a statistically valid degree.

Hall, 61, a former ranking official in the public schools of Newark, N.J., and New York City, took over Atlanta’s struggling system 11 years ago and quickly declared success. She became a prominent figure in the education reform movement even before winning national superintendent of the year honors in 2009. She chairs the advisory board of Harvard University’s Urban Superintendents Program and is chair-elect of the board of the Council of the Great City Schools. This spring, President Barack Obama named her to the National Board for Education Sciences.

Hall also has become a popular speaker on the education-conference circuit, trumpeting what she calls Atlanta’s “trajectory of improvement.”

Her critics cite no evidence that Hall personally altered or concealed data to elevate her national profile. And, in the interview last week, she said her honors were based on more than CRCT scores.

“While we sit and look just at the scores, when you walk into those schools, those schools are transformed dramatically from the schools that were here 10 years ago,” Hall said. “The people who make the decisions about these awards walk in and they see. ... They’re not looking for ultimate victory at this point. They’re looking for progress, real progress.”

It is clear, however, that questionable statistics helped define Hall as a tough, results-driven leader. Put another way: If the performance of Atlanta students had been reported accurately, would Hall still be considered a miracle worker?

“While I want our children to look good, the way to look good is to do good,” Perdue said in an interview last week. Educational progress “has to be genuine, it has to be honest, it has to be transparent, it has to be sustainable. To be sustainable, it has to be real.”

Hall said she takes offense, professionally and personally, at any suggestion that progress by Atlanta students is not valid. Underlying all the district’s efforts, she has said, is the notion that even the most disadvantaged child can excel.

“I want to be candid,” Hall said in a speech this summer. “From Day One, my team and I decided that we would not allow demographics to determine destiny.”

The AJC’s investigation, however, suggests otherwise.

At each of the 12 schools where district officials acknowledge cheating on the 2009 CRCT was most likely, at least 90 percent of students were poor enough to qualify for a free or reduced-price lunch; at two of those schools, every student qualified. At all but one of the 12 schools, at least 97 percent of the students were black; at the other, the figure was 92 percent. Any illicit attempts to increase scores at those schools, the newspaper’s investigation found, may have caused the greatest harm to the students who needed the most help.

Early warnings

“I want to believe. But the numbers don’t make sense.”

From its opening lines, Mark Musick’s letter to the Atlanta school district in 2006 exhibited a friendly, if pointed, skepticism. As a former president of the Southern Regional Education Board, a public school advocacy group, Musick had positioned himself as a supporter of the Atlanta system. His children graduated from Atlanta schools. But the steep, rapid increases in the graduation rate that Hall was citing left him baffled.

Musick’s letter recounted how the graduation rate rose 29 percentage points, from 43 percent to 72 percent, in just three academic years. From 2004 to 2005 alone, it jumped almost one-third. Yet the number of graduates dropped.

“Let’s say that again,” Musick wrote. “The Atlanta high school graduation rate jumped in one year by nearly one-third — an almost unheard-of percentage increase for a large school system — but the number of students who graduated was less than the previous year.

“You do not have to be a math whiz to know that something unusual ... mathematically possible ... but unusual is going on here.”

Hall responded with a polite letter that acknowledged his concerns, Musick said in a recent interview.

But, he said, Hall neither offered an explanation nor indicated that the district would seek one.

The district’s research director, Lester McKee, said recently he had studied the graduation rate “well in advance of Mark Musick’s concerns.”

Other suggestions of anomalies in measurements of student achievement have generated little public reaction by school officials.

In 2001, for instance, dozens of Atlanta schools posted surprisingly large gains on the CRCT. At 30 schools, passing rates increased 30 or more percentage points in one or more test subjects. At 10 others, the gains were at least 40 points.

At M.A. Jones Elementary, for instance, 88 percent of fourth-graders passed in math — a 54-point leap from the previous year’s 34 percent. Of the nine schools with the biggest gains in 2001, seven were flagged by the state for suspicious increases in 2009. The other two had closed in the meantime.

When the 2001 results were released, education researchers asked the district to explain the increases. Some called for independent verification.

The district refused. The data, officials said, needed no analysis.

‘Data Every Day’

The American Educational Research Association had never presented its distinguished public service award to a school superintendent — until this year. The organization honored Beverly Hall for her “inclusion of education data as a predominate feature of her strategies for promoting school reforms and increasing student achievement.”

Hall accepted the award at the group’s annual meeting in Denver, where she delivered a lecture tailored to her audience: “Using Data Every Day, Every Way to Transform Atlanta Public Schools.”

When she arrived in Atlanta, Hall said, according to her prepared remarks, “studying the data and making it transparent to all were widely resisted propositions in our school district.”

“Today,” she said, “these practices are embedded in the organization’s culture.”

She described the “data walls” of charts and graphs tracking student achievement in schools, district offices and her own conference room. She talked about the “teacher effectiveness dashboard,” an electronic tool that measures, among other statistics, students’ test scores while estimating the degree to which specific teachers contributed to students’ progress.

Such data, Hall said, guide every decision.

“Before conducting an intervention,” she said, “we conduct or, in many cases, commission a thorough review of our current performance indicators, collecting data from everything including standardized assessments, scaled questionnaires, surveys, interviews and focus groups, and observations.”

Among the “measurable results” she cited: higher scores on the CRCT.

“Every year since 2000,” Hall said, “across the curriculum and in all grades, students have posted steady academic gains” on the CRCT.

Hall spoke on May 2. She didn’t mention that, not quite three months earlier, state education officials had ordered an investigation into suspected cheating on the CRCT in most of Atlanta’s elementary and middle schools.

In August, after the investigation implicated 108 educators as possible cheaters, Hall dismissed suggestions that her data analysis should have detected irregularities.

“There is no way for the system to know that,” she told an interviewer on radio station WABE. “What should we have known? We can’t yet say there was pervasive cheating.”

‘Below basic’

On the NAEP math test last year, fourth-graders were asked to solve this equation: What is 301 minus 75?

The test offered four possible answers: a) 226, b) 235, c) 236 or d) 374.

Just 49 percent of Atlanta fourth-graders chose “a.” Fourth-graders did worse in only three other large cities: Detroit, Cleveland and Louisville, Ky.

Such gloomy findings appear throughout reports on the most recent NAEP tests. Atlanta’s fourth- and eighth-graders, the two groups tested, scored lower than national averages and lower than their peers in cities with populations of at least 250,000.

On the fourth-grade reading test, for example, 34 percent of students nationwide performed at a level NAEP describes as “below basic proficiency.”

Fifty percent of Atlanta fourth-graders lacked those basic reading skills.

Hall has publicly focused not on actual scores, but on the degree to which they have risen. As poorly as Atlanta fourth-graders performed in reading on last year’s test, the scores had increased at a greater rate since 2002 than those in comparable school systems. Most of the increase, however, occurred by 2007, NAEP reports show. Since then, NAEP officials said, improvements on that test segment have lacked statistical validity.

But if progress continues, Hall said last week, “we will eventually close the gap with the state and the nation.”

A top NAEP official described Hall’s representations about Atlanta’s test results as generally accurate, although lacking in nuance.

“In general, they do better in terms of gains,” Peggy Carr, associate commissioner of the National Commission on Education Statistics, the federal agency that administers the NAEP, said in an interview.

Of Hall’s assertions that Atlanta leads the nation, Carr said: “I don’t think that’s fair across the board. If it’s a general, blanket statement, I don’t think you can say that.”

Hall fiercely defends Atlanta’s showing on the NAEP.

“If indeed what was happening on the state test is a result of cheating, it would show up on the NAEP,” she said in the interview last week.

Other irregularities, however, could easily go undetected.

‘Warehousing these kids’

Through most of the school year, students trickle from other Atlanta schools into Forrest Hill Academy, an alternative school for grades six through 12 on the city’s south side. On April 21, 2008, however, Forrest Hill’s sixth-, seventh- and eighth-grade classes accepted 18 new students.

The same day, Atlanta began administering that year’s CRCT.

This was no coincidence, said Randall Richardson, chief executive of Community Education Partners, the Nashville company hired to run Forrest Hill from 2002 to 2009.

“A ton of kids always got sent to us right before test time,” Richardson said in a recent interview. Principals and other administrators, he said, knew that district officials insisted on holding Forrest Hill, not the students’ home schools, accountable for poor performance.

Two years ago, according to state data, other schools sent 53 middle school students to Forrest Hill in the month leading up to that year’s CRCT, which is given to students in grades one through eight. In the comparable period last year, 26 middle-school students came to Forrest Hill. The transfers sent the school’s enrollment up as much as 11 percent.

“It accomplished the purpose they wanted,” Richardson said. “It got disruptive kids out. It got the low performers out. I understand the pressure on school districts. ... If you’ve got a place to put them where those scores are attributed, that’s a great temptation.”

The students, however, moved into one of the worst-performing schools in the state.

On last year’s CRCT, 97 percent of Forrest Hill’s sixth-graders failed science and 93 percent failed math. The best results were in eighth-grade language arts, where 42 percent failed.

By 2008, academic performance at the school had become so poor, violence so pervasive and disciplinary actions against students so arbitrary that the American Civil Liberties Union sued the school district and Richardson’s company. The suit estimated 30 percent of Forrest Hill’s students were sent there for reasons other than disciplinary infractions. The district’s rules, the suit said, required no explanation for these “administrative referrals.”

“It’s one thing if they are separating out these students who genuinely need extra services or special services and they are actually providing those services,” Emily Chiang, one of the lawyers who filed the case from the ACLU’s headquarters in New York, said in an interview. “The problem is when they are just warehousing these kids.”

A top district official said students come to Forrest Hill from across the city, not just from a few schools trying to manipulate CRCT scores.

“That does not happen,” said Kathy Augustine, the deputy superintendent for instruction. “It just doesn’t happen.”

Whatever the reason for their transfers, many Forrest Hill students are encouraged to withdraw from school even if they cannot enroll elsewhere, said Craig Goodmark, a lawyer with Atlanta Legal Aid who represents students and parents in disputes with the district.

It is not uncommon for middle school students at Forrest Hill to be 16 or older — old enough, under state law, to quit school. If they never enter the ninth grade, they are not counted as dropouts and do not suppress the district’s graduation rate. Goodmark and other lawyers who have represented Forrest Hill students said school officials have pressured students to leave and not return — and they don’t.

“Low performers, disruptive and disabled students are being administratively pushed out to the educational periphery,” Goodmark said. “There is a bull’s eye on those kids’ backs.”

This is the logical result, he said, of Hall’s incessant emphasis on test scores and other performance metrics.

“They don’t do anything but focus on those tests,” he said. “That’s not teaching. That’s just cooking your books to give the appearance of progress.”

The district severed its relationship with Richardson’s company after settling the ACLU suit. Today, it operates Forrest Hill — surrounded by fences, its gate watched by a uniformed security officer — with perhaps even less scrutiny than the school received when it was privately run. But the principal, Tricia Rock, said on the school’s website that Forrest Hill is committed to achievement.

“STUDENT SUCCESS!” she wrote in red capital letters. “WHATEVER IT TAKES!”

Then, in regular print, a disclaimer: “as long as it is legal, moral and ethical.”

‘We’re on top!’

At the University of Delaware last fall, Hall described her work in Atlanta as guided by moral imperative.

“It was clear,” she said, “that we had to move past old stories and old stereotypes and do nothing less than total district transformation.”

She continued: “One of our principals put it best when she described her work today as ‘debunking the American algorithm that socio-economics predicts academic success.’”

Hall has continually highlighted success in schools where it may not have been expected — and where, in some cases, it may not have been real.

In September 2008, Hall took the stage at F.L. Stanton Elementary, on Martin Luther King Jr. Drive near I-20, with a special guest: Margaret Spellings, then the U.S. secretary of education. Spellings came to designate Stanton as a No Child Left Behind-Blue Ribbon School, and to praise Hall.

The superintendent, Spellings said, was a trusted adviser, a “national leader” who was “moving the needle” for progress.

The Stanton students chanted: “Our school is great, and we’re on top!”

Sixteen months later, state officials said they suspected cheating on the 2009 CRCT in 83 percent of Stanton’s classrooms. The district’s own investigation included Stanton on a list of 12 schools where cheating was most likely to have occurred.

In June, more bad news: Under close monitoring, Stanton’s scores on this year’s CRCT plunged. In just one school year, the failure rate among Stanton’s students in some CRCT subjects tripled and quadrupled. The proud moment of September 2008 seemed so long ago.