Part IV of this week’s Five Principles for Smarter Data Systems series–a guest post from Dr. Heather Weiss, Founder and Director of the Harvard Family Research Project (HFRP):
Education Sector’s five design principles powerfully reframe the conversation about how, when and where to use data to support student learning so that it will not die in vast data warehouses, but will live to guide the daily actions of teachers, families, afterschool and summer programs, tutors, and ultimately of students themselves as they assume responsibility for their own learning. While the principles are not now widely implemented, they are not just pie in the sky either—real communities, districts and schools are working on them and there is much to be learned from their efforts.
We are documenting early lessons from them in a set of papers about cradle to career data pathways soon to be available at the HFRP site. Our documentation shows that data systems incorporating the five design principles and accompanied by strong tools and outreach for families, afterschool programs and others can increase the chances of effective family engagement as families understand that the pathway to college begins in early childhood and that they, along with teachers and others, can help students stay on track through K-12 with the aid of data in the accessible form of GPS performance navigation system-type tools. At the cradle end we are finding early childhood programs providing regular developmental feedback and related ways families can support learning thereby creating the demand for actionable performance data when children start school.
Some of the most innovative K-12 work is going on in New York City with the Department of Education’s ARIS Parent Link and New Visions’ 9th Grade Parent Involvement in College Readiness initiatives. Both efforts are piloting and learning from their substantial outreach efforts to engage low income schools and families in getting and using data to support learning in and out of school. Each provides attendance and selected performance data and ARIS conveys grade level and subject area expectations as well. Their experience underscores the need for creative and persistent parent coordinators and other staff dedicated to this work at the school and district levels.
New Visions has created two powerful tools, the School Data Snapshot for teachers and individual Student Tracker for families, students and others. Both track performance on key benchmarks including attendance courses, credit accumulation, and grades necessary to graduate and to be college ready. The Tracker is a planning and performance progress tool informing parent, teacher and student conversations and triggering efforts to stay on track, including credit recovery. New Visions co-designs their outreach efforts with families, creates concrete ways for them to support student achievement, and is beginning to share data with afterschool programs as well. A case study of their work and how it supports whole school college readiness efforts will be available on the HFRP web site.
In a pilot effort to stimulate low income family and student use of ARIS Parent Link in 24 schools in Lower Manhattan, the NYCDOE schools are trying different outreach strategies Early lessons learned include the importance of a welcoming space for parents with access to computers and training in how to use them, the value of utilizing parent and student volunteers (in one school students train their parents and bring back notes confirming they logged their parents in), the importance of training teachers to ensure they are comfortable using ARIS and in having data-driven discussions with parents, and of principal leadership and of persistence and creativity (at another school the librarian opened early for ARIS workshops and at another a parent coordinator emailed parents who had not logged in).
Better Benefits: Reforming Teacher Pensions for a Changing Work Force
The Course of Innovation: Using Technology to Transform Higher Education 
09/03/10 | Kris Amundson gives Virginia policymakers a true/false quiz on the condition of K-12 public education in the state.
09/02/10 | Elena Silva compares two proposals for the next generation of computer-based assessments funded by the U.S. Department of Education.
09/01/10 | Forrest Hinton explains why consumer choice and market forces are currently weak influences on higher education quality. 
09/01/10 | Chad Aldeman's and Kevin Carey's thoughts on value-added teacher evaluations are featured in The Atlantic
I had to read it twice because my mind wondered off for some reason. I think it is because of lack of sleep. I do think you for taking the time to post this.
Right, the data must be accurate. That’s a big issue with current practice. And proactive steps must be taken to ensure proper governance and security. But, let’s think expansively about ensuring that data is accurate. While we need external verification — especially for high stakes items — we also need use. As we note in the five principles, “Importantly, educator use will ensure that data is more accurate, allowing for better information and decision-making at all levels.” To elaborate, it’s the systems where data is collected, never seen, locked away in some warehouse, and then spit out where all sorts of errors creep in. If students, families, teachers — those close to the data — actually use data, they will update it, ensure it is accurate and we’ll have a much more continuous process of checks/balances.
With all due respect, there’s one imperative fundamental precondition for all of the above re class, school, district and state data systems, i.e., that the data be accurate and reliable. From searching widely at the national level and below, it has become unfortunately clear to me that much of the data used to make both grand and fine policy decisions is neither. In 1996, the NYS Comptroller recommended that the NYS Ed. Dept. require LEA attendance and enrollment data to be audited for validity: the recommendation was rejected. A decade ago, after a national audit project, the USDOE OIG recommended that USDOE eliminate certain special ed. school “exiting” categories inasmuch as kids reported in them were actually dropouts. The recommendation was rejected. The NYS Comptroller reported in a recent “audit” (read “investigation”) that NY school districts, which grade their own students’ end-of-course high school Regents exams substantially and inappropriately inflate those grades – and that the NYS Ed. Dept. knows all about the grade inflation, but does nothing. This not only skews high school course completion numbers, but since passing these courses and exams is required for a high school diploma, significantly inflates high school graduation reports as well. Thus it is unsurprising that while NY’s high school graduation rate is up, its NAEP and SAT scores are not.
The IT industry has a succinct phrase to describe what’s going on here: GIGO – Garbage In/Garbage Out. It doesn’t matter how detailed or sophisticated your data warehouses and data analyses systems are if the underlying data is … garbage.
By all means go for highly sophisticated data collection and data analysis systems and further their widespread use in policy and program decisions. Nevertheless, if we don’t insist – stringently – that the numbers pushed into such systems are legitimate and verified, we may just be presiding over creation of the largest GIGO dump in American educational history.