Lesson Observations Are No Way To Grade Teachers

by Nick Morrison, Forbes.com:  http://www.forbes.com/sites/nickmorrison/2014/03/19/lesson-observations-are-no-way-to-grade-teachers/

Teachers teach, so it seems natural to assume that watching them teach is the best way to find out how good they are.

But while it may be taken for granted, a growing body of evidence suggests that observing lessons is no way to grade a teacher.

Research on both sides of the Atlantic has highlighted discrepancies between judgements made by observers and empirical measures of how much progress students make.

In fact, a more accurate correlation between the two could be achieved by flipping a coin.

These findings cast doubt on the role of lesson observation in passing judgement both on individual teachers and on schools. But it does not mean observations themselves are pointless. Instead we should rethink the way they are used so they become an effective tool of school improvement.

The case against lesson observation as a routine part of school inspection is made in a thoughtful report by U.K. think-tank Policy Exchange, published this week.

Its particular target is observations as practised by Ofsted, the school inspection body for England, which tend to last no more than 20 minutes and form part of its overall judgement on a school’s effectiveness.

These inspections have long been the target of anxiety among teachers and concern over their use has forced Ofsted to insist that individual lessons are not graded, despite many teachers’ experiences to the contrary.

The high-stakes nature of school inspections - principals in one county have been told they can expect to lose their jobs as the result of an unfavorable report - ensures they remain contentious.

But the verdict from Policy Exchange is that these observations “are neither valid nor reliable”, and, what is more, they are unnecessary when schools carry out their own - rather more thorough - assessments of teacher performance.

This is partly based on the work of Professor Robert Coe of the Centre for Evaluation and Monitoring at Durham University, who has analysed data from a series of studies on lesson observation, including the Measures of Effective Teaching (MET) project in the U.S., funded by the Bill and Melinda Gates Foundation and involving 3,000 teacher volunteers.

His findings, outlined in a blog posted earlier this month, suggest that in the best case scenario, two observers will agree about the quality of a lesson only 61% of the time.

Agreement is less likely for lessons at either end of the scale. If one observer judges a lesson to be inadequate, the likelihood that another observer will come to the same view varies from 38% to just 10%. These are not great odds when someone’s career is at stake.

Even more worryingly, not only do observers disagree but their judgements are inaccurate when set against measures of how much progress students make in class. Coe concludes that the likelihood of an observation matching this verdict is just 49%. In other words, random chance would be a better predictor of teacher effectiveness.

This reinforces findings from a U.S. study published last year, where observers were asked to rate teachers who had proven ability to raise student achievement. No matter how experienced the observers, they were unable to identify successful teachers.

There are many reasons why this should be the case. Some of those advanced by Coe include the observer’s bias towards particular teaching styles, the fact that it is hard to tell by observation whether learning is taking place and that by focusing on particular things observers can miss much of what is going on.

But this does not mean lesson observations are useless. They play a useful role in performance management, when used in conjunction with other tools, such as measures of progress and student feedback. More importantly, they are also a valuable part of continuing professional development for teachers.

As I reported last month, a study on Shanghai’s education system published by the National Center on Education and Economy singled out lesson observation as a cornerstone of its approach.

Teachers can expect to be observed 20-30 times a year. Experienced teachers welcome observers into their classes. Lesson observation is an embedded part of teacher training and development.

Rather than attempting to use it as a way of judging teachers, we should look at it as a way of helping them to develop. It is only be recognizing the limitations of lesson observation that we can fully appreciate its strengths.