The Waiver Wire: Proceed with Caution

Flickr photo by skyloader

The Center for American Progress released the first analysis of the initial 11 ESEA waiver applications that moves beyond the contents of each waiver proposal to the merits of a state’s application. As noted here, and here, and here the report calls out Massachusetts and Tennessee for submitting particularly well-crafted proposals and identifies four states where significant clarification is needed. These identifications are based on two aspects of the 100+ page applications – the state’s evaluation and accountability plans. Some of the more qualitative portions of the waivers, like how the state plans to transition to college- and career-ready standards and to implement effective school turnarounds, do not factor into the CAP ratings as “stand outs”, “middle of the pack”, or “needs more detail” states.

Specifically, CAP examined the waivers based on four factors:

• clarity and ambition of the state’s AMOs – not too easy, not too hard, but just right;
• number of factors included in school-rating systems (again, CAP – like Goldilocks – preferred a just right amount);
• rigor of the state’s teacher and principal evaluation plan, especially the weight placed on student learning; and
• capacity of the state to implement their evaluation plans, including the state’s data system and whether legislation would be needed.

While I agree that Massachusetts and Tennessee should be lauded for the clarity of their goals and their data capacity, I think it’s also important to recognize that their applications aren’t perfect.  Massachusetts, which has been noted both for its high student achievement rates and large achievement gaps, seems to be moving in a worrisome direction with its proposal to combine subgroups of students into one large “high needs subgroup.” (If you really have a lot of time on your hands, you can read each state’s waiver request here.) The super-group would include students with disabilities, current and former English language learners, and low-income students, but not racial minorities. This could be a good thing if low n-sizes prevent individual subgroups from factoring into accountability determinations. But, aggregating subgroups could also mask uneven performance amongst the individual groups – a school’s special education students may be faltering, while other subgroups in the high needs category shine. If these groups are combined for the sake of setting achievement targets and identifying schools for accountability purposes, it will be more important than ever to transparently report results for individual subgroups, including racial minorities, at the school and district level. And hopefully educators will take action to improve the learning outcomes of students in individual subgroups even if they aren’t required to do so by their accountability status.

Tennessee also tried to go the “simplify” route. The state will focus on improving overall student achievement and on narrowing gaps between groups of students. Sounds great, but the devil is in the details. For achievement targets, the state will focus only on 3rd and 7th grade math and reading, in addition to an aggregate of scores across grades 3-8 (not all schools serve grades 3-8, so I assume this could be an aggregate of grades 3-5, 6-8, etc.). In high schools, the Algebra I and English II exams will be used, in addition to graduation rates. That’s a fairly narrow set of metrics and very focused on standardized tests. Even more worrisome, the state is only establishing performance targets for all students, not subgroups. That means there is no graduation rate target for low-income, special education students, etc. The lack of achievement AMOs by subgroup is supposedly addressed in the state’s gap closure AMOs. But these gaps are only going to be measured on the aggregate 3-8, Algebra I, and English II scores. Graduation rate gap closure targets were not proposed, nor gap closures for individual grade-level tests. Again, since the state is proposing to use only some of the currently-reported measures for accountability, it will be critical for Tennessee to continue reporting specific data for each individual subgroup and assessment administered.

Will it be confusing to parents (and even policy-wonks) to have one set of indicators reported and a different set used for accountability purposes? Most likely, and communications efforts will be needed to ensure that stakeholders understand which measures are reported, which are used for achievement targets, and which are used to determine if schools need interventions. In some cases (like Tennessee), this could creates three accountability systems. Many states noted in their waiver requests that it is confusing to have competing federal and state accountability systems. But it may be just as – or even more – confusing to have multiple state systems.

This is why the most important contribution of the CAP report isn’t calling out individual states, but rather its final recommendations for the Department of Education: 1) don’t rush to approve every application, 2) ask for more information, and 3) proceed with caution. Given what I’ve read in the waiver requests, I couldn’t agree more.