–More from our new report, Putting Data Into Practice…
Students in New York City schools interact with a number of public agencies and participate in a variety of out-of-school activities. All are likely to influence academic achievement. Yet educational data systems are mostly blind to students’ lives outside the classroom. At the same time, public agencies and community groups know little about the academic situations of the youth they serve.
As the school district is doing with ARIS, New York City is working to consolidate disparate bits of data to get a more complete view of the citizens it serves. The city wants to coordinate services across nine city agencies, including those dealing with public health, homelessness, and juvenile justice. On average, a single family is involved with five different agencies, says Linda Gibbs, deputy mayor for Health and Human Services, “but they didn’t know about each others’ presence in the household.” Social services data is not yet integrated into ARIS. Nor is ARIS data available to the social workers who use a new system known as HHS-Connect.
Because of these gaps, educators and community leaders lack data to help them understand how these systems interact and to help them make decisions and coordinate their work. It means they can’t take advantage of powerful tools to detect patterns or risk factors across interventions — patterns that might be impossible to discern from school data alone. This cross-agency information is particularly important to serving at-risk youth, such as children in foster care, who are most likely to use multiple public services.
But in New York and elsewhere, schools and social service agencies are slowly improving their capacity to share data toward useful ends. Some examples:
- In St. Louis, the main objectives of Big Brothers and Big Sisters of Eastern Missouri are to improve attendance, behavior, and classroom success (“ABC”). But the organization had little information about what actually happened in their mentees’ classrooms. So, after getting privacy waivers from parents, officials secured access to school data about students’ attendance, tardiness, behavior, and grades. Now youth workers and mentors can step in, working with parents to improve attendance, for example, and checking up on homework assignments if they see students going off course. They can also recognize and praise student success.
- In California, the Youth Data Archive, run by the John W. Gardner Center for Youth and Their Communities at Stanford University, uses data from schools and community organizations to research questions, such as the association between Boys and Girls Club attendance and the development of proficiency among English language learners.
- Nationwide, Intelligence for Social Policy (ISP) promotes the development and use of integrated data systems. Now working with nine jurisdictions, ISP grew out of Philadelphia’s Kids Integrated Data System, which houses data that helped shape the city’s dropout prevention initiatives. Dennis Culhane, one of ISP’s two principal investigators, says that four of the ISP sites include some educational data and that “all wish they did.” Integrated data, he says, is especially important across developmental transitions. For instance, during early childhood, he says, the “baton gets dropped all the time.”
Still, these examples are nascent. And, without careful attention to the design of data initiatives, the potential to coordinate actions across the variety of organizations and adults supporting youth learning will go unmet.
Click Image To Enlarge
5/29/2012 | The New York Times discusses Mary Nguyen's report "Degreeless in Debt: What Happens to Borrowers Who Drop Out."
5/28/2012 | The Washington Post discusses Mary Nguyen's analysis of college dropouts and loan defaults from her Chart You Can Trust, "Degreeless in Debt: What Happens to Borrowers Who Drop Out." 
5/24/2012 | Education Week discusses Anne Hyslop's views on Mitt Romney's education plan.
{ 3 comments }
Bill,
We have to look forward and backward to the nature of featherless bipeds. Catch 22 says it best. In Catch 22, the purpose of sacrificing bomber crews was producing tight bombing patterns is because tight bombing patterns look good on the cover of Life Magazine. The purpose of NCLB data is is make schools look good or look bad depending on your political situation. If NCLB-type data turns out to be accurate, its a coincidence. The purpose of good data systems, however, is to provide accurate information to improve services.
Data-DRIVEN accountability numbers will always be corrupted. You yourself cited the reason in the bottom of your comment. Fear and protecting turf will always corrupt accountability systems. That being said, accountability is one part of education, just as it is one part of life. We just don’t want the corrupted accountability data to corrupt our diagnostic data. Its like health care. We know that hospitals, insurance companies, etc. will use and abuse their economic power. That’s life and that politics. We wouldn’t want doctors to falsify blood pressure tests. Real world, we know to check our wallets when Boeing talks about the need for weapons systems. But there has to be a fire wall. We don’t want Boeing engineers to falsify engineering stats. In the peacetime army, there is the right way the wrong way and the Army way. In battle, you want officers to pass on honest info.
Only in the disrespected profession of education would “reformers” deny the arguments for firewalls between politicized data and real data.
By the way, did you read the NYC performance audit? NYC manufactures data as brazenly as Texas. Is there any conceivably way that educators speak truth to the power of Klein?
Your report helped convince me of the power of these data systems. I’d like to hear more about the garbage in, garbage out nature of the test scores. As long as NCLB data is used for accountability purposes, there will be pressures to dummy down tests or to do the opposite. But there is no incentive for better tests. I bet there are a lot of honest reformers who would like the RttT to incentivize better assessments. But those individuals will be scattered around sysyems and they will be going up against deep-seeded institutional forces.
I don’t want to sound like a cynic. I’m not a cynic. I can see data-driven accountability for hard sciences, such as checking weight-bearing loads of bridges. Back in the day, the accounting profession had a culture that celebrated their green eye shades mentality, and besides, their numbers required interpretation but they were still rooted in reality.
But have you ever heard of a field similar to education where data-DRIVEN didn’t corrupt the data? Had you ever seen a central office using data-DRIVEN accountability where their numbers were not bogus? Have you ever seen a data-driven accountability system where honest information was allowed up the chain of command?
John, thanks for the kind words on the report. A few responses to your questions:
**On accountability, NYC’s overall strategy puts accountability together with autonomy and enabling tools and support (perhaps someone from NYC could describe this in more depth). I think that’s the right conceptual approach — to create a reciprocal obligation. The implementation, getting the right measures, and a million other factors are important; it’s obviously not simple and a place for disagreement, but conceptually I don’t see the two as necessarily opposed. We need to be smart about how both types of info are important. From the report: “But, because it varies so widely across classrooms, this micro-level data is usually not included in the accountability programs that states use to compare school performance. This is despite the fact that micro-level data is critical to improving day-to-day student performance, says Beverly Donohue, vice president of policy and research at New Visions for Public Schools, a school support organization responsible for working with 76 of the district’s public schools. The issue, Donohue says, “is not accountability versus micro-level student data.” Both types of data are valuable, but serve different purposes.”
**As to sharing data, some of this as you note is good old turf protection, some a reflection of isolated work, etc. There’s also a great deal of fear and misunderstanding around federal privacy laws (FERPA), something that the administration would be well-suited to help clarify. Finally, many times these are excuses because it takes more time and effort to figure out how to make something work than to just say no. These things are present in a variety of circumstances — it’s a leap to draw the line directly to testing as a cause.
**Finally, it’s important to think forward, not backward. We should applaud the Obama administration, with state consortia leadership, for the big bet on much better assessment tools and practices. As I’ve blogged, this is going to be difficult. There’s broad agreement that our assessment tools are not adequate. The question is whether this new endeavor will meet its potential.
Brilliant study. You’ve laid out all of the positive dynamics of digital miracles and the need to get out of silos, address whole students and whole groups of students, collaboration, and turning teachers into learning scientist.
I’d like to hear more about a couple of challenges. Your report is very consistent with the research of Rothstein and other that data-driven, data-formed decision-making is inherently incompatible with data-DRIVEN accountability. How has the NYC push for accountability interected with the constructive use of data systems?
Secondly, how will districts move ahead now that their priority is to expand primitive old NCLB testing to meet “reforms” imposed by the Obama administration? Hasn’t the proliferation of new testing mandates produced a poision pill that will delay better assessments and better data systems until these RTTT and VAMs systems get abandoned? In the meantime, it might problem be best and double spending. Go ahead and create a garbage-in garbage out system for firing teachers and fabricating data, and create a second system that isn’t poisoned by “accountability” in order to help kids.
Thirdly, I’m curious about the conservative policies on sharing data. When I started teaching, teachers were allowed to know which students were categorized as ADHD, but now we’re not. I know IDEA changed, but our district’s interpretation seemed too conservative. Now, teachers in my district can not know if our kids are being treated for mental illness unless the kids or a parent tells us. I think I’m told of those problems more than other teachers. As teachers become more focused simply on instruction, I worry that they are becoming less sensitive to the kids’ problems and kids feel less willing to share. As Larry Cuban puts it, accountability and testing has created more “social distance” between teachers and students. We need a culture where data is used to bridge those distances.
And that gets back to breaking down silos. Unless we’re told, teachers can’t know that a kid is just out of the lockup, where he’s been taught to walk out of class if angry, off his anti-pyschotics, and can’t get to the block where his mother could sign a form due to family and gang conflicts. I could repeat dozens of those examples.
Anyway, great report.
Comments on this entry are closed.