School choice and charter school advocates increasingly contend test scores provide a limited snapshot of a school’s effectiveness, maintaining other factors may be more telling, including graduation rates, college attendance and parental satisfaction. They also argue more allowances ought to be made for the characteristics of the students, who, in many urban schools of choice, are low-income and start kindergarten already behind suburban peers.
If those comments sound familiar, it’s because traditional public-school educators have been saying them for years. They, too, have been pleading, “Judge on us on more than test scores.” And they, too, have pointed to the challenges of educating low-income students, transitory students, students who come to school hungry and tired.
In a recent report on how to best assess school choice including charters, the American Enterprise Institute concludes: “Test scores should not automatically occupy a privileged place over parental demand and satisfaction as short-term measures of school choice success or failure.”
The study, "Do Impacts on Test Scores Even Matter? Lessons from Long-Run Outcomes in School Choice Research," says test results don't deserve the import that policy now accords them, stating:
A growing number of studies are finding that school choice programs can improve high school graduation rates, college attendance, and earnings —without producing gains in test scores. Conversely, studies of other school choice programs have found large short-term test score gains but no lasting benefits in terms of graduation rates or college attainment. Improving test scores appears to be neither a necessary nor sufficient condition for improving the later-life outcomes that truly matter.
Our findings beg serious questions about using standardized tests as the exclusive or primary metric on which to evaluate school choice programs. If test score gains are neither a necessary nor a sufficient condition for producing long-term gains in crucial student outcomes, then current approaches to accountability for school choice programs are questionable at best. Our findings suggest that focusing on test scores may lead authorities to favor the wrong school choice programs. Focusing on test score gains may lead regulators to favor schools whose benefits could easily fade over time and punish schools that are producing long-lasting gains.
But, in a recent review of that American Enterprise Institute study, Christopher Lubienski of Indiana University and T. Jameson Brewer of the University of North Georgia fault the conclusions, saying
The report's suggestion that achievement scores should play a smaller role in determining the efficacy of school choice models represents a stunning effort to move the goalposts in search of new justifications for supporting their preferred policies. After decades of pro-school-choice research and advocacy promoting test score comparisons with public schools as the primary measurement for evaluating school choice models (e.g., charters and school vouchers), the AEI report now suggests that less attention be given to these learning outcomes… after decades of idolizing standardized achievement outcomes and promoting their use as the best way to measure and evaluate school choice options, they contend that test scores are actually just an "interim measure" and weakly related to more important outcomes later in life. But, instead of analyzing the correlation of achievement measures in school choice programs with later life outcomes, the report focuses on arguing that achievement and attainment measures are not highly correlated, at least in the school choice studies they collected. The report concludes that policymakers' primary focus should not be on the negative effects that school choice has on achievement scores.
How should we judge schools?
Is parental satisfaction – a criteria being pushed by the charter sector – a reliable gauge?
Research shows parents often value things other than core academics in their child’s school, from the marching band to the soccer program to arts offerings. When DeKalb County was considering closing schools that suffered both low enrollment and performance, parents turned out in force in their school colors to defend their school. They were not concerned with the “F” ratings for academics given to their schools based on state test scores. They told the taskforce that the principal greeted each child by name, the school held mom and muffins breakfasts and their children loved their classmates and teacher.
Is high school graduation a better gauge than test scores of whether a school is a success?
A new Georgia State study commissioned by the State Charter Schools Commission found attending a Georgia startup charter high school increases the likelihood of high school graduation by roughly 4 percentage points and that such students are 6 percentage points more likely to enroll in college than comparison groups of students who attended charter middle schools but then switched to traditional public high schools.
The students who attend a charter high school are also about 8 percentage points more likely to persist in college for two consecutive semesters and 2 percentage points more likely to earn a college degree or certificate than those who went to a traditional high school, according to the study.
The authors acknowledge that other studies of charter schools in Georgia haven’t shown an overall improvement in test scores, noting:
These results lead to additional questions for research…charter schools in Georgia have shown mixed results in improving students' test scores. Some scholars attempting to identify the skills and qualities imparted to students by charter high schools claim that such schools can teach hard-to-measure qualities such as grit, persistence, self-control and conscientiousness. While these skills are not fully captured in test scores, they are important in achieving long-term outcomes. Additional research is needed to determine how and why Georgia start-up charter schools are improving life outcomes for their students
Speaking of judging schools, Georgia's first online statewide charter high school closes next month, a victim of performance deemed below standard by the State Charter Schools Commission. But Monica Henson, leader of Graduation Achievement Charter High School, told the AJC the commission's performance bar was unrealistic for a school like hers. As the AJCs Vanessa McCray reported today.
The school's data shows that more than 60 percent of its students are over-aged for their grade; the average ninth grader is nearly 17 upon enrollment. Henson points to other challenges: 3 percent of current students have been incarcerated, and the school serves small numbers of students who are in foster care, pregnant or parenting, or were rescued from sex trafficking.
Henson argues the formula doesn't account for high-risk factors for which the state does not collect data and said the measurement does not take into consideration that schools with "abnormal student populations may perform differently than those with typical student populations."
Your thoughts on how best to evaluate schools?
About the Author
