Backed by a group of business and political leaders, the Common Core State Standards for English and math were released in June 2010. Since then, more than 45 states have committed to using the guidelines on what students should learn to shape instruction in their classrooms.
Two groups – or “consortia” – of states formed to create new standardized tests that would measure Common Core skills and knowledge. The goal was to design exams that were more in-depth and precise than most states’ existing testing programs. The consortia also hoped to provide students, teachers and schools with scores that could be compared across state lines.
Both the Partnership for Assessment of Readiness for Colleges and Careers (PARCC) and the Smarter Balanced Assessment Consortium plan to launch their tests in 2014-2015. PARCC received $186 million and Smarter Balanced received $175 million from the federal government to create the tests.
Pockets of resistance to state standardized tests — even in traditionally pro-testing states such as Texas and Florida — cropped up across the country this spring as students sat for the yearly exams.
Some advocates even urged parents to “opt out” and refuse to allow their children to participate. Repeated errors by testing companies were among the chief complaints.
And yet, largely lost in the public conversation about the country’s aggressive testing regimen is talk of how to make quality-control an ongoing priority.
The concerns come with testing in flux nationwide as most states revamp the exams to measure knowledge of national Common Core standards.
But new tests are likely to face many of the same challenges that an Atlanta Journal-Constitution investigation found bedeviled the old ones – a breakneck race to get scores returned, a lack of state personnel to provide oversight, and a high volume of near-simultaneous test-takers. The tests will face new hurdles as well, experts say, such as greater reliance on technology.
In general, advancements in testing have not only ushered in improvements, but also temporarily heightened the risk of error, said Ellen Haley, president of CTB/McGraw-Hill. “That’s when things can happen that you didn’t expect,” she said.
Already, work on the Common Core exams by two “consortia” of states has taxed state departments and the testing companies involved, and complaints have flared about cost.
“Everyone’s work level essentially has doubled over the last couple years,” said James Mason, Mississippi’s director of student assessment. “You’re trying to build a new airplane while flying your own.”
States that have bowed out of the common tests, including Georgia and Indiana, will face similar issues, whether they create new tests alone or partner with smaller groups of states. Some will work with a smaller budget than the states that belong to consortia.
Across the country, states’ testing programs have suffered for years from quality-control failures that, at times, threatened to hurt the students they were supposed to help. Thousands of pages of documents obtained by The Atlanta Journal-Constitution detailed more than 100 testing snafus and revealed flaws in questions on many exams – even as states used them to make key decisions about schooling.
Whether the new consortia of states will address such problems is an open question, said Thomas Toch, of the Carnegie Foundation for the Advancement of Teaching. Doing so would cost more money, he said.
Representatives of the groups said they plan to offer tests that are both more sophisticated and less prone to error.
“I’m not saying we have all this locked down at this point,” said Jeff Nellhaus, director of assessment policy research and design for the Partnership for Assessment of Readiness for College and Careers (PARCC) coalition.
But, he said, the group recognizes that steps such as scoring need a high level of oversight to make sure results are accurate and comparable across states. Test questions will undergo careful analysis. Audits and redundant checks will be built into the process.
Member states will still have some oversight responsibilities, he said, but they will also have PARCC’s support. “Over the course of the next two decades there might be a little problem every once in a while,” he said. “We’re going to minimize it.”
The Smarter Balanced consortium will offer states a common pool of rigorously reviewed questions and access to software proven to work, said Executive Director Joe Willhoft. States will also undergo certification that their tests met key standards from a prominent testing research center at the University of California, Los Angeles.
Scoring, however, will remain the states’ job. They will hire their own contractors and develop their own plans for oversight through measures such as reviewing analyses of question responses.
The consortium is trying to learn from the experiences of states that had a better track record with testing, Willhoft said. “There were a lot of states doing really good things,” he said. “It’s just, not all states were doing good things.”
Oversight board proposed
Industry officials and experts point to progress made over the years to reduce testing mishaps through measures such as the publication of a “best practices” guide and federal peer reviews of state programs.
Those efforts, however, don’t regularly scrutinize how well the exams actually perform in a given state. Nor do they examine testing company contracts or handle complaints about problem tests.
Such gaps have led some testing experts and observers to advocate creating an oversight board for testing.
Toch is among them. In a 2006 report, he argued for a national testing oversight agency. Such an agency could audit state testing programs and function like federal consumer-protection agencies, he wrote.
“These experts would in an unbiased way, with an independent perspective, take the measure on an ongoing basis of the profession to reduce the amount of error,” he said in a recent interview.
The goal: To make sure the standards for the tests were as high as the expectations for the students taking them.
The idea is promising, said Chris Domaleski, a senior associate at the National Center for the Improvement of Educational Assessment, as long as it preserves the flexibility states need to carry out their programs.
“I don’t hear any of my colleagues in the field objecting to more scrutiny and oversight,” he said. “Any sensible professional in the field would welcome it.”
The proposal also gets support from Karran Harper Royal, an activist and parent of two in New Orleans. Testing errors have helped erode public confidence in the exams, and she said an independent board could address some of those concerns.
“If we are going to base so much on tests,” she said, “there needs to be an independent entity that evaluates the tests to make sure they are of the highest quality.”
Such a measure would likely need to overcome some industry resistance, however.
Stuart Kahl, a co-founder of testing company Measured Progress, said talk years ago about such an external auditing agency went nowhere. “My problem is I find it very difficult to believe that any outside group would be knowledgeable enough to do a good job of it,” he said. “I just don’t think that will solve anything.”
Others also have doubts. Jon Cohen, executive vice president and director of assessment at the American Institutes for Research, said the real motivator for testing companies is competition in the industry.
“Oversight boards are not typically that effective because there is so much nitty-gritty to get into,” he said.
Fines as deterrent
There is one step states can take now to increase pressure on the industry to deliver better quality control. That is to include in new contracts potential fines for errors.
Only about 20 states used such a power during the past decade, according to states’ responses to information requests from The Atlanta Journal-Constitution. Some, such as Vermont, said they had no reason to. But others didn’t specify in their contracts the power to seek compensation for errors.
Kahl said bad publicity and damages clauses in contracts push companies to get better. “Testing companies – they don’t like penalties, they don’t like bad press,” he said.
States must negotiate such provisions with testing companies. Texas quietly dropped from its contract with Pearson last year a phrase that made it easier for the state to seek damages when the company made a mistake. Then-Education Commissioner Robert Scott and his chief-of-staff were unaware of the change.
“We would never approve that,” said Scott, who left the department later that year after criticizing testing. The deletion put the state in a precarious position, he said.
Texas education officials said that while the contract change deleted some legal language, it still allows the state to seek reduced payments to Pearson when problems arise.
‘A big mistake’
A final avenue for addressing quality-control problems on tests, some experts say, may simply be to revisit policies that force schools and districts to use test scores to make critical decisions.
“I think we’ve created an environment where we want to replace human judgment with results from tests,” said Kahl, the testing executive. “And that’s a big mistake.”
Peter Behuniak has devoted his career to testing. A former testing director for Connecticut, he’s now a measurement professor at the University of Connecticut.
And yet, his advice in recent years has taken on a cautionary tone.
“A lot of people think that the solution to whatever problem they are focused on is another test,” he said. “More is not necessarily better.”
About the Author
