Alfie Kohn, Data Denier

There’s a real need for healthy skepticism around our nation’s quest to collect and utilize education data to improve and deepen student learning. Complex formulas, such as those used to calculate value-added scores for teachers, need to be open to examination, testing, and improvement over time. Policymakers and educators need to better understand how to interpret and use assessment data, both from statewide summative tests and their own classroom activities. And we need smart policies, practitioners, and even skeptics to help us use better information about student learning to its full potential.

But there’s a big difference between healthy skepticism and denial. Alfie Kohn, writing in Education Week, sits firmly in denial.

While it’s healthy to remember that quantitative evidence should be used to inform complex decisions or activities — not as a total replacement for human judgment — that’s not Kohn’s point. Kohn doesn’t want to improve the use of data, because that effort would require an acknowledgment that objective evidence exists and can be used to make judgments. Instead, he wants to construct and knock down straw men.

For Kohn, any hint of standardization, even a rubric that might imply common expectations across multiple classrooms, is to be resisted. Evidence about learning is subjective and based entirely on individual judgment:

You’d object to any procedure that seems mechanical, in which standardized protocols like rubrics supplant teachers’ professional judgments based on personal interaction with their students. And the only thing worse than “benchmark” tests (tests in between the tests) would be computerized monitoring tools, which the reading expert Richard Allington has succinctly characterized as “idiotic.”

Later, Kohn gives an example of what he does like. And, here, his false dichotomy makes it clear that his understanding of data is limited to the straw man he’s constructed:

It focused on teachers’ personal “connection[s] with our subject area” as the basis for helping students think “like mathematicians or historians or writers or scientists, instead of drilling them in the vocabulary of those subject areas or breaking down the skills.” In a word, the teachers put kids before data.

In Kohn’s mind, data and deeper learning are polar opposites. There’s no way to generate evidence, other than looking into the white of Joey’s eyes, to know whether or not Joey is thinking like a mathematician.

In reality, teaching Joey to think like a mathematician has everything to do with the use of data. The only real way to help students think like mathematicians is to understand, describe, and when possible, quantify what that thinking really looks like in practice. Take, for example, the assessment experts developing epistemic games, using fancy computers and crazy things like quantitative statistics to give educators and students much better information about progress. And while those experts are in general not big fans of the current state assessments used for NCLB determinations, they’re not anti-data, anti-computer, or anti-evidence. These are not necessarily the same things.

Teaching is incredibly complex. But, it is not the mystical craft that Kohn would have us believe.