SAT Score Hysteria and the Missing Chart

Here’s the headline and lead paragraph from today’s front page Washington Post story (emphasis mine):

SAT reading scores drop to lowest point in decades

SAT reading scores for graduating high school seniors this year reached the lowest point in nearly four decades, reflecting a steady decline in performance in that subject on the college admissions test, the College Board reported Wednesday.

From The College Board’s press release:

It is common for mean scores to decline slightly when the number of students taking an exam increases because more students of varied academic backgrounds are represented in the test-taking pool. However, a decline in mean scores does not necessarily mean a decline in performance. There are more high-performing students among the class of 2011 than ever before.

While the Post did mention the changing demographics further down in the story, they omitted this key chart:

Score Declines May Not Mean What They Seem

The College Board also reports that:

• The number of college-bound seniors who benefitted from SAT fee waivers increased nearly 77 percent since 2007 (possible proxy for low-income students); and
• Among SAT takers in the class of 2011, 44 percent were minority students, making this the most diverse class of SAT takers ever.

When reporting test scores, it’s essential to understand whether the scores reflect changes in performance or changes in the tested population (see here and here for how this plays out with NAEP results). In this case, while I don’t have enough data to know exactly what is going on with SAT scores, there’s no doubt that the story is more complex than meets the eye.

UPDATE: Commenters, please help me give an award for the most egregious claim made by an interest group about the 2011 SAT scores.

My nominee is FairTest, a group which spends much of its time arguing all the reasons the SAT is flawed. The anti-standardized testing organization had to work hard to develop a press release which used test scores as the basis for their claim, but still disavowed test scores. Here’s the nomination: “Large 2011 SAT Score Decline Shows Failure of “No Child Left Behind.”