Guest Post written by Craig D. Jerald of Break the Curve Consulting.
On Friday the Bill & Melinda Gates Foundation released the latest findings from its $45 million Measure of Effective Teaching (MET) Project. MET researchers concluded that teachers should be evaluated using “multiple measures” of teaching effectiveness, including student surveys, value-added estimates based on student test scores, and direct observations of classroom practice. Indeed, the MET Project recommended that evaluation systems should include observations even though they are expensive to conduct reliably and add little to the predictive power of value-added scores alone. Despite facing very tight budgets, not one single state or district will ignore that recommendation and evaluate teachers without directly observing them in action.
The reason for that costly consensus is that observations contribute something that test-score based measures simply cannot. As the MET Project put it, “the real potential of classroom observations is their usefulness for diagnosis and development of instructional practice.” In other words, everyone agrees that teacher evaluations should be about more than simply measuring performance; evaluations also should be about helping educators improve their performance. And it’s worth spending more on a set of evaluation measures that can accomplish both.
Yet many education advocates who wholeheartedly support the use of multiple measures for teacher evaluation still resist such measures when it comes to evaluating schools, including the school-level analogue of classroom observations: on-site “inspections.” In a paper published by Education Sector today, On Her Majesty’s School Inspection Service, I use examples from England to show how inspections can contribute the kind of diagnostic feedback and support for improvement nearly completely lacking under No Child Left Behind.
England is just one of dozens of countries around the world where inspectors visit schools. Just as most U.S. policymakers would never consider evaluating teachers without observing them, those countries would never consider holding schools accountable without taking time to inspect them.
Of course, U.S. advocates who worry that inspections might “water down” school accountability systems do have a point. Frankly, I shared similar concerns when Education Sector invited me to write a report on this topic. But after spending months studying England’s approach, I became convinced that there are ways to incorporate inspections into school accountability systems in ways that do not sacrifice “rigor.”
In fact, I concluded that England’s current inspection system offers clues to solving many challenges related to holding schools accountable in the post-NCLB era, including the following:
- Judging schools on a broader range of evidence without losing sight of the fundamental importance of student achievement, including standardized test scores;
- Leveraging expert judgment rather than relying solely on quantitative formulas, yet still ensuring sufficient safeguards against inconsistent or inflated ratings; and perhaps most importantly
- Achieving a better balance between rigorous evaluative ratings and better diagnostic feedback to help schools improve.
For all of those reasons, state and federal policymakers should give careful consideration to the potential for using inspections to improve school-level accountability systems. It’s time to think about “observing” schools, too.
Click Image To Enlarge
5/29/2012 | The New York Times discusses Mary Nguyen's report "Degreeless in Debt: What Happens to Borrowers Who Drop Out."
5/28/2012 | The Washington Post discusses Mary Nguyen's analysis of college dropouts and loan defaults from her Chart You Can Trust, "Degreeless in Debt: What Happens to Borrowers Who Drop Out." 
5/24/2012 | Education Week discusses Anne Hyslop's views on Mitt Romney's education plan.
{ 3 comments }
It is wonderful to see school inspection back in the national discussion. It offers a whole new world to American school accountability.
Jerald introduces it well. His study is clear and identifies several of the key contributions that English Inspection offers American accountability designers. The most important of these is the value and role of professional judgment in inquiry about school performance.
But what it misses is also critical if the American discussion is going to go anywhere. In the American discussion the notion of “expert judgment” was introduced in 2008 by an American researcher advocating site visits a la England. “We need an elite corps of professional experts to visit schools.”
But, the evidence for this conclusion had not been observations of English inspectors at work. It was something of inferred evidence that went like this. His researchers did observe a very limited set American accreditation team visits. They found that these visits lacked sufficient rigor. Since accreditation teams were comprised of school people serving as volunteers, they decided, “We can’t use amateurs; we must build a national corps of trainned experts.”
Jerald’s claim that English inspectors exemplify this special breed is not very penetrating of English views on the qualities and knowledge sets for a good inspector. It sounds like London OFSTED officials talking to an American.
Inspection began in England in 1839. The first qualification in the early going and now is considerable experience doing good teaching. Yes, of course there is training as we know it. But there is also a set of the mentored apprenticeships that are common to the development of practicing professions in many fields. The base of the expertise needed to make accurate conclusions about the quality of complex action in a classroom comes from experiential knowledge of how practice actually works.
In the end, evidence that paper qualifications of team members is what determines a rigorous visit that prepares legitimate conclusions is not persuasive. Much more directly related, particularly in American settings, is the rigor of the protocol for conducting the visit and for writing conclusions.
The 2nd Jerald miss was Rhode Island’s SALT school visit initiative. I played a key role in the design and implementation of the SALT visit from 1996 to 2008.
I was asked to do that by the Commissioner of Education. I had just completed a year long study of English inspection, observing a hundred or more English inspectors at work in English schools. This study was the center of my Teachers College Press book, Reaching for a Better Standard: English School Inspection and the Dilemma of Accountability for American Public Schools. Reviewers on both sides of the Pond saw the book as authoritative and provocative.
The SALT group built a visit protocol based on what I had learned in England. Starting as a small pilot, the SALT visit was successful beyond all of our expectations and became a system that could complete visits of every Rhode Island school once every five years.
I displayed a number of conclusions written by SALT teams at an international conference, that included HMI inspectors. While they agreed with the critical importance of actual classroom experience, many were a bit snobby about using “ordinary” teachers on a part time basis. (Chairs were full time.) The HMIs present exclaimed, “Amazing. These conclusions are more penetrating and better written than our conclusions.”
By the end of 12 years Rhode Island invested over 6 million dollars in state dollars in the SALT visit. We visited all except a handful of Rhode Island public and charter schools; several more than once. Over 2,000 Rhode Island teachers and principals had served on teams. Several evaluation reports were prepared including a on-site review of the process by two British inspectors. SALT was repeatedly studied, and the historical document record is unusually complete. (For more detail on SALT go to http://catalpa.org/proj_salt.cfm.)
The most important contribution the SALT experience could make to the current discussion is to provide the grist so that the early discussion could shift from abstract speculations about how to organize a visiting enterprise to first questions.
What do we want to learn about schools? How can we learn that with rigor and certainty? Can a visiting methodology provide that? How would doing that help improve school practice? AND then how do we organize it.
That would intensify the chances that the discussion of English inspection would last a while and contribute to our successfully building a more effective system of accountability for American schools.
Both Craig Jerald and I are right. English inspection does provide major provocations and new possibilities for much improved designs for American school accountability, professional development of teachers and strategies for school improvement.
Gideon, you’re quite right that many U.S. charter school authorizers inspect schools. Ultimately, I chose to focus on England’s model because charter inspections tend to be for a narrower purpose, charter renewal, and I thought England’s approach more relevant to the broader accountability debate. As I mention on page 3 of the report, “Some states require charter schools to undergo reviews by visiting teams before their charters can be renewed. Massachusetts consulted with experts familiar with Ofsted’s approach in order to model its charter reviews on English inspections, and the state publishes each school’s ‘renewal inspection report’ online. However, so far the state only inspects charter schools, and the inspections play no role in regular accountability determinations.” But for states considering inspections, it would be worthwhile to speak with some of those charter authorizers.
No one seems to have noticed that some high quality charter school authorizers do use regular inspection visits as part of their high stakes accountability systems. They even post their findings for the public to read. The key is to have clear standards for school performance and focus on quality programs rather than compliance monitoring. They often use external experts to augment their inspections teams, providing targeted expertise and objectivity.
Comments on this entry are closed.