Here’s the headline and lead paragraph from today’s front page Washington Post story (emphasis mine):
SAT reading scores drop to lowest point in decades
SAT reading scores for graduating high school seniors this year reached the lowest point in nearly four decades, reflecting a steady decline in performance in that subject on the college admissions test, the College Board reported Wednesday.
From The College Board’s press release:
It is common for mean scores to decline slightly when the number of students taking an exam increases because more students of varied academic backgrounds are represented in the test-taking pool. However, a decline in mean scores does not necessarily mean a decline in performance. There are more high-performing students among the class of 2011 than ever before.
While the Post did mention the changing demographics further down in the story, they omitted this key chart:
Score Declines May Not Mean What They Seem
The College Board also reports that:
- The number of college-bound seniors who benefitted from SAT fee waivers increased nearly 77 percent since 2007 (possible proxy for low-income students); and
- Among SAT takers in the class of 2011, 44 percent were minority students, making this the most diverse class of SAT takers ever.
When reporting test scores, it’s essential to understand whether the scores reflect changes in performance or changes in the tested population (see here and here for how this plays out with NAEP results). In this case, while I don’t have enough data to know exactly what is going on with SAT scores, there’s no doubt that the story is more complex than meets the eye.
UPDATE: Commenters, please help me give an award for the most egregious claim made by an interest group about the 2011 SAT scores.
My nominee is FairTest, a group which spends much of its time arguing all the reasons the SAT is flawed. The anti-standardized testing organization had to work hard to develop a press release which used test scores as the basis for their claim, but still disavowed test scores. Here’s the nomination: “Large 2011 SAT Score Decline Shows Failure of “No Child Left Behind.”
Click Image To Enlarge
5/11/2012 | U.S. News & World Report highlights ideas from "Moving On Up: How Tuition Tax Breaks Increasingly Favor the Upper-Middle Class." 
5/11/2012 | In Diverse, Steve Burd comments on the Obama administration's plans to address student debt. 
5/11/2012 | Kevin Carey talks to Salon about the government's role in rising student debt.
{ 3 comments }
Matt: No. You are misinterpreting what I’ve written and then taking those misinterpretations to extreme places. I’m merely pointing out that the media, and the public at-large, need to be informed consumers of data. In particular, we need to make sure to understand the information prior to making both conclusions (i.e., performance dropped) and assigning causes to those conclusions (SAT test scores are down because of NCLB). Basic statistics says that you must be careful about making inferences about CHANGES in performance if the group you are comparing with is dissimilar to the initial group tested. In this case, we have evidence that the comparison group is dissimilar, so it is important to be cautious in making conclusions.
The above has nothing whatsoever to do with having high expectations and believing that all students can learn. And, talking about CHANGES in performance over time is very different from discussing absolute performance.
Pointing out that changing demographics has a relationship to average scores, couldn’t that be construed as racism. Shouldn’t we hold all our various demographic groups to the same high standards.
Next thing you know you are going to be telling me that we need to stop illegal immigration in order to fix our schools, and that since Whites and Asians in America out perform Whites and Asians in almost every other country our school system must be doing really well!
via Twitter, @KenMLibby nominates http://blog.coreknowledge.org/2011/09/15/same-sat-different-day/
Comments on this entry are closed.
{ 2 trackbacks }