Milwaukee Vouchers are Only a Failure if By “Failure” You Mean “Failure”

Rick Hess offers this spin on the newly-announced failure of the Milwaukee voucher program to improve student learning:

[The study found that] when it comes to test scores, students with vouchers are performing no differently than other kids. (It is worth noting that [voucher] students are being educated more cheaply than are district school students). What to make of the results? First off, 20 years in, it’s hard to argue that the nation’s biggest and most established voucher experiment has “worked” if the measure is whether vouchers lead to higher reading and math scores. Happily, that’s never been my preferred metric for structural reforms–both because I think it’s the wrong way to study them (see “Science and Nonscience“) but, more importantly, because choice-based reform shouldn’t be understood as that kind of intervention. Rather, choice-based reform should be embraced as an opportunity for educators to create more focused and effective schools and for reformers to solve problems in smarter ways. Whether any of that pays off is much more a question of quality control, support, talent, investment, infrastructure, and the rest than it is of whether or not a choice program is in place.

I’m having a hard time understanding the distinction here. Vouchers aren’t a teaching method or curriculum or instructional intervention. They are, as Hess notes, purely structural, shifting control over resources toward parents and widening the range of institutions that can receive those resources. Since “more focused and effective schools” are properly defined as “schools where students learn more” i.e. “schools with higher reading and math scores,” if vouchers didn’t result in more such schools then vouchers failed. One might argue that vouchers created the opportunity for educators to create such schools and educators didn’t take advantage of it, but what’s the difference? The whole point of structural reform is to change incentives and conditions; if the change was insufficient to create desired behavior then ipso facto the reform failed. A purely structural metric for evaluating purely structural reforms misses the point altogether.

Hess then suggests that vouchers (or school choice programs more generally) are better understood as a necessary but not sufficient condition for improvement that also requires “quality control, support, talent, investment, infrastructure, and the rest.” Perhaps, although it’s worth remembering that those are things that can be effectively applied to the cause of school improvement without vouchers. All in all, I think this lends credence to my theory that you either try vouchers for real or you don’t. Half-measures, of which even the Milwaukee vouchers are an example, will never produce satisfactory evidence either way.