Study challenges use of test scores to gauge teacher effectiveness

Even with student growth measures, researcher finds teacher ratings fall when they move to higher-poverty schools
A Portland State University researcher and former middle school math teacher says, “There is this idea that teachers in high-poverty schools are just not trying hard enough. The teachers in the wealthier schools were patting themselves on the back for their scores, but their jobs were a lot easier. I could have done nothing, and the kids I taught in the wealthy school would have been successful.”

A Portland State University researcher and former middle school math teacher says, “There is this idea that teachers in high-poverty schools are just not trying hard enough. The teachers in the wealthier schools were patting themselves on the back for their scores, but their jobs were a lot easier. I could have done nothing, and the kids I taught in the wealthy school would have been successful.”

Dara Shifrer’s past experience as a middle school math teacher and her research now as a college professor on educational inequality led her to delve into the use of test scores to assess teacher effectiveness.

Her conclusion: Who and where someone teaches plays a key role in being deemed highly effective.

A sociologist who examines social inequality and educational outcomes, Shifrer tracked the value-added scores of 4,500 teachers and found their scores rose when they switched into high-performing (often low-poverty) schools and fell when they switched into low-performing (often high-poverty) schools. Her study utilized scores from the Education Value-Added Assessment System, used in a dozen states, including Georgia.

Value-added measures use student growth as measured by tests to analyze how much a teacher advanced the learning of each student. A 2013 Georgia law required student results to be at least 50% of each evaluation, but the percent was reduced to 30% in 2016.

Because teachers are judged on how much growth their students show rather than whether the students earn a high score, the argument has been that valued-added measures don’t depend on whether the class is full of top achievers. The contention is that teachers shown to be able to improve student performance should be able to do so in any classroom in any school.

Shifrer found that was not true. If teachers transferred to a school with more poor students, English language learners, or tougher teaching environments, then their value-added scores dropped. “The changes in individual teachers’ scores over time were consistent with shifts in the economic status and race of students in their classrooms and schools,” said Shifrer.

Shifrer’s study adds to an already broad research base impugning the reliance on student scores to grade teachers and schools. “Policymakers and legislators want something numerical and objective, or that at least seems objective,” she said.

Yet, there is no denying the socioeconomics of school performance, easily seen when you note the highest-performing schools in Georgia are almost always those serving affluent areas.

Before earning her doctorate and joining the faculty at Portland State University, Shifrer taught at middle schools on opposite ends of the income spectrum. “There is this idea that teachers in high-poverty schools are just not trying hard enough,” said Shifrer in a recent interview. “The teachers in the wealthier schools were patting themselves on the back for their scores, but their jobs were a lot easier. I could have done nothing, and the kids I taught in the wealthy school would have been successful.”

The false narrative that a dedicated teacher can triumph over any social ills is dangerous, said Shifrer. She entered teaching full of idealism but found the reality — including a mentor advising she may only save one student a year — disheartening and ultimately defeating.

“I envisioned coming in and being this creative teacher,” she said. “I loved teaching, and it broke my heart when what I envisioned wasn’t possible. I had all these ideas, but I was totally tamped down. No Child Left Behind was kicking into gear. I was given a curriculum that I had to read from; the kids hated it, and I hated it. I was just a cog in the wheel. I had to show up every day and regurgitate this material that was not effective."

The teachers who choose to work in schools serving struggling communities deserve greater respect, said Shifrer. “Right now, we are demonizing them. Teachers working in the hardest contexts are getting the most pushback and blame and no respect for what they are doing.”

Portland State University professor Dara Shifrer

icon to expand image

What would Shifrer tell discouraged teachers?

“Don’t base your worth on the performance rating of your school or of you. As much as I found teaching a challenging job — much more challenging than going to grad school or being a professor — I still look back at it fondly. It was the most meaningful job I ever had. Even if schools and teachers can’t totally eradicate the effects of societal inequality, it’d be a whole lot worse if public schools and teachers didn’t exist,” she said.

Will the pandemic influence how we view teachers?

“Honestly, I experienced some glee as I watched parents and families register what it meant for schools to close — the collective realization of how much our society revolves around schools and how hard teachers actually work,” she said. “Schools not only build kids’ cognitive and social knowledge but facilitate parents’ ability to work and essentially are a substantial piece of the village raising our children. And then once remote learning started, parents, even those with relatively high levels of educational attainment, struggled to assist their children with course material and to manage their children’s attention and behavior.”

And what about the anger that some parents are expressing now when teachers oppose demands to resume face-to-face classes?

“I think that speaks to the attitude we maintain toward teachers, that we expect people to come to the teaching field because they’re called or have a vocation. Because they care — the feminization of the profession — and that this should suffice in place of respect, status and compensation,” she said. “That they even should be willing to risk their mental health, and now physical health, because their passion to help the children is so strong. I remember every time I’d tell someone I was a middle school teacher, they’d cock their head to the side as if to say, ‘Well, aren’t you a good sweet person.’ Why would we structure a fundamental social institution on such nebulous and unfair expectations?”