The problem with using tests to rate music, art and gym teachers


Other states do it

Georgia is one of about 20 states using student academic growth as a major factor in rating teachers. In Georgia and other states, new teacher evaluation systems were part of applications for federal Race to the Top grants.

How good is your 8th-grade band teacher?

Georgia’s new teacher rating system bases about half of teachers’ job ratings on an administrator watching them teach and half on their students’ academic growth.

For example, to measure “student growth” in an 8th-grade band class, students might take a three-part test at the start of school and again in April. The test could include playing two major scales, a sight-reading exercise and a multiple-choice test.

About half of the teacher’s overall job rating depends on how much test scores from students in their classes improve from the start of school to the end.

Source: Georgia Department of Education sample student learning objective statement

To read more about Georgia’s new teacher evaluation system, check out MyAJC.com. You’ll find recent stories on a plan in Fulton County to recruit high-performing teachers to struggling schools and an examination of how the state’s new evaluation system could affect teachers of lower-income students.

This year, all Georgia teachers — not just those who teach core subjects — will be rated in part on student test results for the first time.

That’s straightforward enough for core teachers whose students take state standardized tests. But the majority of teachers – in subjects like art, music and gym – teach subjects and grades that aren’t covered by such high-stakes tests.

For them, many school districts have come up with their own exams. But educators and research suggest this approach isn’t good enough for evaluations that could make or break careers.

The new system for rating these teachers is open to cheating, educators say, because in some cases teachers administer and grade the very tests used to evaluate them. The quality of tests varies by district, meaning a Spanish teacher in Gwinnett could be graded differently than one in Atlanta. And there are concerns about fairness, because research shows teachers of non-state tested subjects tend to score lower than those who teach courses where state standardized tests are given.

Georgia Department of Education deputy superintendent Avis King said the department is aware of the concerns and taking steps to address them.

“That’s why we are being very careful and cautious as we move forward,” she said.

The state’s plan is part of a new educator evaluation system which bases about half of teachers’ job ratings on an administrator watching them teach and about half on their students’ academic growth.

For teachers of grades and subjects covered by state tests, including math, English, social studies and science, students’ growth is measured by state tests.

For about 70 percent of teachers, whose areas are not covered by state tests, it’s often measured by tests their own districts design.

The new system could change. Georgia's incoming state school superintendent, Richard Woods, has said test scores should play a smaller role in teacher evaluations. And Georgia has asked the U.S. Department of Education for a delay in using the new overall ratings for decisions about hiring, firing and pay. But they'll still be used this year to determine which educators in 26 districts receiving federal Race to the Top money get millions of dollars in bonuses. Educators have told the Georgia Department of Education there are problems with how teachers of non-state tested subjects are evaluated, state reports on districts already using the new system show.

The tests and the cut-off scores that place teachers at different rating levels vary from district to district. Some districts — like Atlanta Public Schools — use multiple-choice tests to evaluate all teachers. Other districts combine multiple choice tests with other kinds of tests, like essays or how well music students, for example, play a C-major scale.

The state has sample materials to help guide districts in setting goals for student growth in different classes.

Carrie Staines, a teacher at Druid Hills High School in DeKalb County, said the quality of test questions in her district is poor. She should know: She was among the DeKalb teachers who volunteered to help write them. The Advanced Placement psychology test she wrote with two other teachers is far too short, at 20 questions, and reflects only “random” tidbits of knowledge that isn’t necessarily crucial, she said.

State officials say the new system isn’t supposed to be used to compare teachers in different districts. The idea is to measure how much students “grow” in every classroom, said Michele Purvis, an evaluation system specialist with the Georgia Department of Education.

“They’re not designed to compare this British lit class in this district to a British lit class in another district,” she said.

Another issue educators are concerned about: Student growth ratings for teachers of areas not covered by state tests tend to be lower than those for teachers of state-tested subjects, according to a 2014 University of Georgia research report.

In some cases, the lower scores could be due to initial miscalculations in districts’ expectations, said King, the state department of education official. “There’s a learning curve involved” with the new tests, she said.

And in some districts, teachers administer and grade the tests that are used to evaluate them. The state monitors its standardized tests in math, reading and other areas for cheating, but security for these new, local tests is left up to individual districts. So far, the number of potential test-security problems reported has been “relatively low,” King said.

But Melissa King Rogers, an English teacher at Druid Hills High School in DeKalb County, said “I think it’s just wide open to the sorts of scandals we’ve seen in APS.”

She was referring to the test-cheating scandal that resulted in the indictment of 35 former Atlanta Public Schools employees and allegations of secret answer-erasure parties and other subterfuge.

The Atlanta Journal-Constitution asked Bill Slotnik, executive director of the nonprofit Community Training and Assistance Center, which has helped dozens of states develop ways of evaluating teachers, if Georgia's method for teachers of areas not covered by state standardized tests is fair and likely to be effective.

“Fairness, like beauty, tends to be in the eye of the beholder,” he said.

But Georgia’s system appears to be running into challenges, he said. Georgia would do better to show educators how a new evaluation system could improve instruction, he said, and involve teachers directly in finding better ways to teach students and reach the goals set under the new system.

“The more these kinds of things don’t happen, the more” the evaluation process “or any other reform just becomes a compliance activity,” he said.

Read more about teacher evaluation systems from previous AJC stories and others at http://bit.ly/1zJH0sQ.