Measuring teachers by the numbers is not easy work.
More than 40 states including Georgia are discovering the challenges as they try to discover which teachers are succeeding and which are not.
Along with being seen as fair and reliable by teachers, these new review systems have to make sense to parents. “There aren’t cookbook solutions to any of these design issues,” said Grover J. “Russ” Whitehurst, the former director of the Institute of Education Sciences within the U.S. Department of Education.
Whitehurst is among the Brown Center on Education Policy at Brookings researchers who released a study last week examining teacher evaluation systems in four urban districts that already have them up and working.
The public debate has focused on the use of student test scores in defining good teaching. But only 22 percent of teachers can be evaluated on student test scores. Most teachers teach courses or grades for which there are no corresponding state standardized tests.
But all teachers are evaluated based on classroom observations, which is often overlooked in the debate. Those classroom observations by principals account for at least 40 percent of a teacher’s final ranking and up to 70 percent.
The Brookings study found teachers who teach high-performing students benefit in observations. The administrators observing them – typically school principals – give these teachers higher marks than teachers of lower performing students.
The study concludes: “Under current teacher evaluation systems, it is hard for a teacher who doesn’t have top students to get a top rating. Teachers with students with higher incoming achievement levels receive classroom observation scores that are higher on average than those received by teachers whose incoming students are at lower achievement levels, and districts do not have processes in place to address this bias.”
According to the study, 9 percent of teachers of the lowest-performing students earned top observation scores. In contrast, 37 percent of teachers of the highest performing students netted a top score.
The rates reverse for teachers ranked in the bottom; 29 percent of teachers in classrooms with low-performing students ranked in the bottom 20 percent. Only 11 percent of teachers of high-achieving students did so.
The assumption has been that classroom observations capture what teachers are doing independent of the students. “But what observations are really capturing is the dynamic of the teacher and student, and that dynamic is influenced by the kind of student you have,” said researcher Dan Goldhaber, a leader in the study of value-added measures of teachers.
Whitehurst and his colleagues offer a solution: Teachers assigned to classrooms with low-performing students should get a sort of “degree of difficulty” boost in their observation ratings. Adjustments for the makeup of the class are already factored into teachers’ scores based on test grades; this would be an extension of that practice.
“Either we have to adjust demographically or we have to come with an observation system that is immune to the effects introduced by different characteristics of students,” said Whitehurst.
Sandi Jacobs of the National Council on Teacher Quality said it has to be clear expectations are not being lowered. “It’s not that the quality of instruction should be lower in a classroom where students are lower performing. In fact, there is a need for even higher quality instruction in such a classroom,” she said.
It’s also critical people doing observations know what quality instruction looks like. In the past, Jacobs said principals gave teachers points for attractive bulletin boards and evenly spaced window blinds.
Today, classroom observations must assess teachers on what really matters. “We’re asking our principals and administrators to do something that’s so important and so central to kids’ learning that they haven’t been asked to do before,” said Jacobs. “We are asking them to go into classrooms and not just say, ‘Well, everything looks okay here.’ Now, we need them to sit across from teachers and say, ‘Yes, it was a good lesson, but it wasn’t a great lesson and here are things you can do to really hit out the park.’”
