First Take: $350m Assessment Competition Guidelines

October 29th, 2009 | Category: Accountability

Last week, the U.S. Department of Education released its initial overview of the competition process for the $350 million in stimulus funding set aside to improve student assessment (see Education Week’s helpful summary).

What We’ve Learned

  • It’s All About Accountability: Not surprising, but important to distinguish that ED’s plans are for assessments related to current NCLB mandates–reading and math in grades 3-8 and once in high school–no science, no early education assessments, primary focus on summative, etc. (more below on ideas to ensure that the investment is leveraged into other areas).
  • Only State Consortia are Eligible: Consistent with the above, this is very much about accountability and in tandem with common standards–school networks or other entities with promising ideas to advance assessment are not eligible to apply.
  • What’s New? Look for teacher scoring of constructed responses, growth and vertical scaling, and openness to both high school end of course exams and testing that does not occur at a single point in time.

What’s Going to Be Difficult?

How can we develop assessment systems for the next decade while constrained by the state of the art from 2001 (NCLB)? For example, there’s huge interest in determining student growth, but if we can only test content at grade level, then we don’t have specific information about students on either end of the distribution–meaning, that it’s difficult to get an accurate gauge on growth.** And, while abandoning every year testing would require significant trade-offs, what if we could trade frequency for quality and depth without undermining strong accountability?

As I wrote about with my colleague Elena Silva, we need to think about this as the first step in a longer term plan to significantly improve student assessment–not just a sweetener on top of common standards or the accountability debate. That means we have to be able to innovate in a world beyond NCLB with these funds. And, it creates a serious co-dependency between this funding and NCLB.

This might be heresy, but really, should we enter into an innovation contest immediately taking these and other ideas off the table? Is there a way out of the current political false dichotomies? Could we carve out a very purposeful and controlled place for innovation just within the projects funded here, avoiding the danger of a free-for-all?.

The Missing Vision

We need to structure this funding so that all states can easily implement the best pieces from the variety of ideas that will be piloted. Without careful attention to the structure behind the funding and whatever is created, at best we’ll end up with something like slightly better state reading assessments in the New England states and better state math tests in the Midwest. Good, but just part of what needs to happen with assessment to improve teaching and learning. We can get much more if we design the program in a way that this major $350m purchase re-sets the assessment marketplace so that whatever is funded can cross consortia lines, is open to continuous improvement and audit, and builds infrastructure for a wide range of both summative and classroom assessment practices.

Bubble sheets, scanners, and a variety of psychometric principles are part of a deep set of standard, shared tools that enable the current methods of assessment. If we want a much better system of assessment, it’s not just content standards that policymakers need to think about. A variety of open standards and infrastructure for student data, modeling, and assessment software platforms also need to be implemented. Just as in content, a lack of open standards and shared infrastructure in these areas is a critical barrier to comparability. Perhaps more importantly, it’s a costly, tangible barrier to the incorporation of new ideas and technologies (more on this).

**NCLB requires that tests align with state content standards and that each student be assessed at his or her official grade level. Lawmakers quite rightly wanted to ensure that test results would be comparable from student to student and create common standards for all students, regardless of their backgrounds. In the paper-based world we couldn’t do this and get information below grade level because we’d have to pass out different tests, with different expectations. That constraint no longer exists in the same way.

Posted by Bill Tucker at 1:54 pm | Tags: , , , , , | 3 Comments

3 Responses to “First Take: $350m Assessment Competition Guidelines”

  1. mosaic lamp says:

    thank you admin supers

  2. Dick Schutz says:

    Q: “How can we develop assessment systems for the next decade while constrained by the state of the art from 2001 (NCLB)?”

    A: We can’t. But it’s a lot worse than that, Bill. The constraints are in test theory that goes back to the early 1900’s when psychologists thought in terms of “latent traits.” That theory was abandoned long ago by everyone except psychometricians who are applying the theory to try to measure the transparent instructional accomplishments of teaching kids to read and do arithmetic.

    Today the “big thing” in the business world is “Business Intelligence.”

    From Wikipedia:

    Business Intelligence (BI) refers to skills, technologies, applications and practices used to help a business acquire a better understanding of its commercial context. Business Intelligence may also refer to the collected information itself.

    BI technologies provide historical, current, and predictive views of business operations. Common functions of Business Intelligence technologies are reporting, OLAP, analytics, data mining, business performance management, benchmarking, text mining, and predictive analytics.

    Business Intelligence often aims to support better business decision-making.[1] Thus a BI system can be called a decision support system (DSS).

    The “skills, technologies, applications and practices” are directly extendable to Educational Intelligence, but they don’t start with filling in bubbles or constructing responses in response to artificially contrived items. We’re pitching $350 down the drain. That’s just a down payment on the billions dedicated to the four propositions of RttT which the National Academy of Sciences has warned are without foundation.

    http://books.nap.edu/openbook.php?record_id=12780&page=1

  3. The world is full of dichomomies that are not false. As much as I learn from your discussions of innovation, I simply can’t comprehend how you expect to move forward on innovative assessments while not backing off from data-driven accountability – or at least accountability that looks, sounds, and smells like NCLB-type accountability.

Leave a Reply