The headlines from today’s Grade 12 National Assessment of Education Progress (NAEP) are grim: reading scores are down a total of four points since 1992, while math scores are up from 2005, the only prior year of testing, and achievement gaps are relatively stagnant. But, given that we’ve focused most of our educational resources and accountability systems on K-8 schools, 
today’s results should not surprise us.
The best way to look at NAEP results across grades is not from single test results that get released sporadically, but to look at the Long-Term Trend (LTT) assessment that’s been in place since 1973. Those results show marked improvement in math and reading at the lower grades, but those gains fizzle out over time. The chart below shows the change in LTT math scores by grade. A 10-15 point gain on NAEP is equivalent to one full grade level, so our nation’s fourth-graders are scoring about two grade levels higher than they did in 1973. For eighth-graders, the progress is about one full grade level. But for 17 year-olds, the progress is only a few months.
So why have we made progress at the lower grades but not in the higher ones? First, the results are simply a testament to how hard educational progress is to maintain. Without following success with further success, students are likely to slip through the cracks. Second, it’s easier to agree on what should be taught at younger grades. It’s much easier to agree on what should be taught in third-grade reading, for example, than what students should be learning in high school English. In high school, expectations for students start to diverge, and some students are slotted into college-going tracks while others are put in vocational classes.
Partly because of this divergence, we haven’t done a good job of coming up with new models for high school accountability. No Child Left Behind essentially takes a K-8 accountability model and plops it down on high schools. Many of the accountability provisions–basic skills tests, allowing students to transfer, offering tutoring, etc.–do not apply at the high school level.
Yet, states continue to move in a sideways direction on measuring success of high schools. They’ve mainly taken the lessons of K-8 school reform–set standards, implement them, then hold high-stakes tests on the content–and tried to expand it to high schools. But, it just doesn’t work that way. High school students have a broader range of things they need to know and be able to do, and low-level standardized tests are not the way to measure those things.
To impose accountability on high schools, about half the states have adopted end-of-course and graduation exams. To show how this is working, consider some of the states participating in the Grade 12 NAEP pilot today with end-of-course or graduation exams. This is the first year of state-level Grade 12 results, but they provide us an interesting lens to compare what states report versus what NAEP found:
- Arkansas is in its first year using its Algebra I end-of-course exam as high-stakes, meaning students who fail to pass could potentially be withheld from graduating. According to the state, 76 percent scored proficient or advanced, while only 5.5 percent scored “Below Basic.” Those student can take the exam two more times and then, if they still haven’t passed, they have the option of using an alternate assessment. Compare that to NAEP, where only 15 percent of Arkansas students scored proficient or advanced, while 41 percent were below basic.
- Idaho says 77 percent of its 10-grade students are proficient or advanced, while only 9 percent are below basic. On the NAEP 12th grade results, only 23 percent of Idaho students are proficient or advanced, and 34 percent are below basic.
- New Jersey says 73 percent of its students are proficient or advanced, while NAEP says only 31 percent are. According to NAEP, 33 percent of New Jersey’s students are below basic in math, yet New Jersey’s high school graduation exam, intended to set a high bar, has prevented less than three percent of students from graduating.
- In Massachusetts, more than 90 percent of students are able to pass their required MCAS scores prior to graduation.
In all of these states, students who fail the first time are given opportunities to retake the test multiple times, receive extra tutoring, or take an alternative assessment. All of these measures help boost the rates. But, while passing the assessments mean a student is eligible to graduate high school, they do not mean the student is ready to succeed in college or a career. To do that, we’re going to have to think differently and actually start to measure college- and career-readiness. Until then, we’re likely to see the same lackluster results at the high school level.
Update: EdWeek went with the “scores are up since 2005″ headline. That’s true, both reading and math are up at statistically significant levels since 2005, but that’s only because reading dipped so far in 2005 that we can see both gains since 2005 and a net decline since 1992. I think the latter is more important.
Click Image To Enlarge
5/11/2012 | U.S. News & World Report highlights ideas from "Moving On Up: How Tuition Tax Breaks Increasingly Favor the Upper-Middle Class." 
5/11/2012 | In Diverse, Steve Burd comments on the Obama administration's plans to address student debt. 
5/11/2012 | Kevin Carey talks to Salon about the government's role in rising student debt.
{ 13 comments }
Bert, Caroline, and Sherman: I think you all are taking the comparison too literally. It’s not that I’m implying that NAEP and state proficiency bars are, or should be, exactly equal. It’s that states have consistently set their proficiency bars at pretty low levels. We see evidence of this in all grades, and the high passing rates on graduation exams are just the latest example.
Bert Stoneberg is right — shame on the original poster for either failing to do adequate research, misinterpreting the research or willfully deceiving. That was DEFINITELY “below basic” work, Chad Aldeman.
By the way, not only are 12th-graders extremely prone to blow off the tests due to senioritis, teen willfulness and increasing awareness of their freedom to do so — it’s the smartest kids who are most likely to do that. Anyone who disagrees is someone with no contact with actual teenagers.
Here’s what Diane Ravitch says about the hoohah about these new NAEP scores:
The reading and math scores for 12th grade students on the National Assessment of Education (NAEP) were just released, and they are unimpressive. Scores are no better than they were in the early 1990s. The achievement gap is unchanged.
I can hear the gnashing of teeth, the cries for more accountability, more charters, more this or more that. But not to worry. In fact, the 12th grade scores don’t mean much. They probably mean nothing at all.
The National Assessment Governing Board (NAGB), which oversees NAEP, has known for years that 12th graders don’t try to do well on the tests. The students know that the tests don’t count, that there are no individual scores, that no one will ever know if they did well or poorly, and they are not motivated to do their best.
The public does not realize that NAEP is a sampling test, and it is not given to every student. They also don’t realize that no student takes the entire test, only a portion of it. The seniors may not know that they are part of a national sample, but they know that this test will not affect their grades, their likelihood of graduating, or their plans after high school.
Unlike students in grades 4 and 8, students in grade 12 understand that their scores don’t matter to their lives.
NAGB has done studies and convened a commission to study the problem of 12th grade student motivation, but has not devised an answer. (To learn more, Google “National Assessment Governing Board twelfth grade motivation NAEP”.)
And that’s why we should ignore the 12th grade NAEP scores. Unlike the SAT, the ACT, the end-of-course exams, the AP exams, and graduation exams, the NAEP tests don’t matter. And seniors know it. They doodle on their test papers, or they select answers with a pattern, like all B, or all C, or ABCD/ABCD. Or they leave questions blank, without even bothering to make a guess.
The government should stop wasting money on this test in this grade, and the usual critics should turn their fire elsewhere.
Chad, Bert is correct in an important sense: there’s nothing consensual about the meaning of either NAEP or state achievement bands. All that you know is that a score in the Advanced band on NAEP is higher than in Proficient, which in turn is higher than Basic. They’re ordinal categories. That’s all.
Rating younger students on basic skills, mostly multiple-choice tests is no more justifiable than doing it for older students. The process drives out curiosity, thinking, creativity, engagement.
On the main NAEP, score gains since the start of NCLB have flattened (reading) or slowed (math) for almost all student groups at grades 4 and 8. Accountability as structured by NCLB is not a sucess, certainly not for reading and questionably for math.
We need to overhaul assessment and accountability – and not how Duncan is doing it. For a real alternatives, see http://www.edaccountability.org as well as my organization, http://www.fairtest.org.
Why is it so hard to see what is really “broken”? When do we look at the beginning level of the process. I want to say that no one is at fault here. The tools we use aren’t as effective as they should be and the only way to see it is to look at an analogy. I lived in Japan for a short time and saw all the kindergarten students arrive on the first day of school already being able to read and write. No one taught this skill, parents simply read stories in the Kana alphabet and the kids memorize the story. Kana is a artificial alphabet it took them 800 years to develop to supplement the Kanji. Kanji are letters they borrowed from the Chinese. Kana letters are like the IPA letters we see in the dictionary. Each character represents the 46 sounds in the Japanese Language. We need to fix our alphabet the same way except it is too late to add new letters to our alphabet. We just need to change the letter names to their best sound…as name for our letters ….and the teach with a simplified English contrived using this simple consistent code for sentences. Which is easier teaching with 10,000 sight words with thousands of spelling exceptions or a 22 sound/letter code?? We don’t see this well because we all learned to read with these “broken” letters. These neural pathways created at the time we start this process determine everything! Like Suzuki teaching music “by ear”. The student’s brain knows the sound that will be produced before his finger touches the key..an auditory task. If the student first learned using notes on a page it would be a visual tactile task..it is not auditory! The letters are like the keys on a piano…they are supposed to talk…only we want our eyes to listen! A symbol system between two senses represent a missing sense…reading is an auditory task…the page is auditory…We can fix everybody in kindergarten if we first fix the effect of our letters on the process skill of syntax! Ralph Parker, Speech Pathologist
Sorry, Chad, but comparing “NAEP Proficient” to “state proficient” doesn’t make sense. Why would anyone want to compare the percentage of students scoring B+/A- or higher with the percentage of students scoring C-/C?
“NAEP Basic” and “state proficient” are different estimates of grade-level performance (which begins at the C-/C level). As pointed out earlier, “NAEP’s policy definition of its ‘Proficient’ achievement level is ‘competency over challenging subject matter’ and is implicitly intended to be higher than grade-level performance.”
I think it would be more useful to compare the percentage of students from different tests in the range of scores defined to estimate grade-level performance. When using NAEP to confirm state test results, the best comparison is “NAEP Basic” vs. “state proficient.”
I do psycho-educational evaluations at a High School in Houston. This is the comment I found most telling.
“Students’ aspirations exceed their skills.”
And it’s totally educators fault. We’ve convinced the students that they must have a 4 year degree. Congrats, now they have the aspirations but will not be able to complete it. Way to go.
The sad fact is that too many or my (our) colleagues in the High Schools do not know what is required to compete at 1st tier Universities. When I am talking to them, I mention my kids schools; if they know what’s going on, they are bowled over. If they don’t; they don’t even recognize the schools. So how can we expect them to establish and maintain the academic rigor required for university level work?
Odds are, they aren’t.
Bert, I don’t think it’s nonsense to point out that states overstate the educational prowess of their students.
Chad, I’m sorry, but I’ve never found nonsense to helpful or important.
Bert, I think the contrast is important. States are inclined to say the vast majority of their K-12 students are performing adequately when external results like NAEP or college remedial rates say otherwise.
“Idaho says 77 percent of its 10-grade students are proficient or advanced, while only 9 percent are below basic. On the NAEP 12th grade results, only 23 percent of Idaho students are proficient or advanced, and 34 percent are below basic.”
This non-sense claim confuses “NAEP Proficiency” and “state proficiency.”
State assessments often define “proficiency” as solid grade-level performance, often indicating readiness for promotion to the next grade. NAEP’s policy definition of its “Proficient” achievement level is “competency over challenging subject matter” and is implicitly intended to be higher than grade-level performance. — Andrew Kolstad, Senior Technical Advisor, Assessment Division, National Center for Education Statistics
You might want to know more about the NAEP Proficient vs. state proficient difference: Stoneberg, B.D. (2007). Using NAEP to confirm state testing results in the No Child Left Behind Act. Practical Assessment, Research & Evaluation, 12(5). Available online: http://www.pareonline.net/pdf/v12n5.pdf
The reading scores are down since 1992, and that’s statistically significant as well. They just dipped far enough down in 2005 that our small progress this year looks good.
According to EdWeek’s coverage of this same topic, the increase of scores from 2005 is statistically significant. http://www.edweek.org/ew/articles/2010/11/18/13naep.h30.html
Comments on this entry are closed.
{ 3 trackbacks }