State system to track students flawed

By Heather Vogell

The Atlanta Journal-Constitution

Despite more than a decade of effort and millions of dollars, Georgia’s system for tracking the progress of public school students remains deeply flawed.

Accurate data on everything from test scores to attendance is widely seen as crucial for understanding why students and teachers succeed or fail, and whether education policies are working.

Yet the development of Georgia’s student data system has endured years of false starts and setbacks that have fueled critics, taxed educators and left state leaders scrambling.

State education officials applauded in April when Georgia won an $8.9 million federal grant to fix shortcomings in the system. But barely a month passed before state Superintendent Kathy Cox said the state would scale back its contribution to the project because of budget cuts.

Earlier this year, a key technology vendor filed for bankruptcy. And the state Department of Education this month hired its third project leader in five years.

Georgia education officials acknowledged the project has hit snags and taken too long. But they said the grant and new hire — a former Georgia Pacific executive — are significant steps toward replacing older, less efficient data collection methods.

That can’t happen soon enough for some.

“We’re quite frustrated that this piece hasn’t been finished yet,” said Stephen Dolinger, president of the education and business group the Georgia Partnership for Excellence in Education.

“It’s literally been years and years and millions of dollars that have gone by,” he said. “We keep hearing we’re going to have a statewide information system and it’s not quite finished.”

Good student data systems help educators spot trends and help schools identify potential dropouts in time to reach them. Georgia has a limited ability to do either, and is instead consumed with more basic tasks of collecting and managing the data.

“We are drowning in data and starved for information,” lamented notes in a draft PowerPoint presentation by the Department of Education obtained by The Atlanta Journal-Constitution.

A source of strife since the 1990s, the student information system has served as a bludgeon for politicians to beat up opponents. It has roiled business leaders puzzled by the state’s failure to accomplish what they say should be a straightforward task.

And the project has baffled one software provider who says the state pays $308,000 a year for software it barely uses that could solve some of its biggest problems.

The reasons for the failings are hard to pinpoint. Some speculate the system wasn’t made a priority, or given enough money.

What’s clear is the project passed through so many hands, it’s hard to know who is most to blame for its troubles. The effort has spanned two state superintendents and three governors, a revolving state board of education and a host of state officials. Cox has wrangled with it throughout her two terms.

At issue is Georgia’s ability to build a data system with “longitudinal” capabilities, which allow educators to track students and their test scores, among other things, from year to year and school to school. It is a simple idea that has been difficult to execute.

This summer, U.S. education officials proposed rules for states competing for key federal “Race to the Top” funds. The grants would stress the importance of a longitudinal system and require states to remove legal barriers to linking students’ test scores to classroom teachers. States’ capacity to measure teachers’ performance will likely be a priority for the awards, federal officials said.

Right now, Georgia’s data system cannot link the test scores of students below sixth grade with teachers.

The state also still loses track of thousands of students a year when they transfer out of their districts or drop out. The AJC reported in June that the state’s data system showed no further records for nearly 20,000 students who were coded as having simply switched school districts last year. The glitch could signal problems with the reported graduation rate, which relies on an accurate count of dropouts.

The holes in the system have left some observers perplexed. A few have questioned the political will to build a system that, if running well, could reveal more about Georgia’s weaknesses.

“Once you have the good data, you can’t hide it anymore,” said Cathy Henson, a former state board of education chairwoman and education law professor at Georgia State. “You can’t say you have a 75 percent graduation rate when everyone else is saying no, not really, it’s really between 58 percent and 68 percent.”

The data project has made some progress. In recent years, all Georgia students have received unique identification numbers to replace less dependable names and Social Security numbers. That was a big step, state officials said.

The state also retooled its older collection methods so they were less burdensome for districts, officials said, and improved access to historical data.

But major limitations remain, state documents show. Data errors abound. Updates are infrequent, which means last school year’s data is sometimes the most recent available. The state also can’t produce a reliable graduation rate using the federal formula it must soon switch to.

Overall, the current information system is so labor-intensive that state workers spend most of their time collecting and managing the data. They have little time to analyze it to improve policy. The state agency hardly has workers to spare, with recent budget cuts slicing into its technology staff.

The problems date to former Superintendent Linda Schrenko, who tried to bring a student information system online in the mid-1990s. But the state soon discovered its vendors were unable to complete the task satisfactorily, said former Deputy Superintendent Bill Gambill.

Gambill, who rejoined the department after the project fell apart, said the state dumped the vendors and set out to try again.

Gov. Roy Barnes created a new agency in charge of the project called the Student Data Research Center. But when he left office, the education department reabsorbed the agency.

“If you can’t guarantee continuity of effort that’s immune from political shifts, you’re just going to keep spinning your wheels,” Henson said.

The state began the project again in 2003 under Cox. By late 2005, state board members were hopeful. “We’ve developed that rocket to launch our students to the next level of learning,” pronounced one member.

But that December, the system blew its deadline for completion, despite repeated promises from the project leader that it was on time.

By fall 2006, more than 80 percent of local school districts still couldn’t contribute their student information to the new system. The state reverted to older, more limited methods.

Department of Education Chief of Staff Stephen Pruitt said that while he can’t speak for what happened then, the state is poised today to move forward.

“Did it go as quickly as we’d like it to? No. Did we have hiccups and bumps in the road? Yes,” he said. “We will have a highly functioning longitudinal data system.”

One vendor says Georgia doesn’t take full advantage of software it already licenses.

Shawn Bay, founder and CEO of eScholar, said the education agency uses less than a quarter of the software’s capabilities.

The software could update data continuously and provide more thorough checks for errors, he said, but Georgia uses it to do neither. The problem: The state’s piecemeal system contains extra layers of technology that Bay said slows data tasks. Other states that use eScholar have simplified their systems and had more success, he said.

“There’s double work going on at every step of the process,” Bay said of Georgia.

Pruitt said that probably a lot of vendors would say their product could do the job. The state is now working primarily with Deloitte Consulting, which bought part of vendor BearingPoint after its bankruptcy. The state plans to end its use of eScholar.

The state chose years ago to build its own system instead of buying one like eScholar off the shelf as a permanent solution, Gambill said, because at the time, Georgia officials believed no product could do everything they wanted.

Now, the state seems to have neither, said Melanie Stockwell, a former education department attorney.

“You ought to have at least one or the other, as much money that’s been put into it,” she said. “We’ve really needed it for 10 years.”

The delays have affected districts’ plans, too. In 2004, the state asked districts to hold off buying their own new data systems, for instance, to make the transition to the state’s system easier. But then years passed.

“We finally decided we couldn’t wait any longer,” said Morgan County Superintendent Stan DeJarnett, whose district bought a system three years ago.

Gov. Sonny Perdue is waiting, too. His office has provided additional oversight through a project review panel. “There’s been a level of frustration with the inability to get the system up and running,” said spokesman Chris Schrimpf. “We’re optimistic and hopeful that over the next year it will be up and running in a usable manner.”

Since the 2004 fiscal year, the state education department has spent at least $14.5 million building the system — not including the new $8.9 million federal grant. More work and money is needed to tie the state’s preschools and public universities into the K-12 system.

James Bostic Jr., a state board of education member, said he hopes to see the system largely completed in about 18 months. He said the department has the right people in place to finish it.

“I’ve had angst about how long it’s taken, also, and the fact that it wasn’t done two to three years ago,” he said. “But the reality was it wasn’t done, and now we have the opportunity to get it done. And I think we’re on the right track.”