It’s relatively obvious to anyone who looks that the Knowledge Is Power Program (KIPP), the nation’s largest charter management organization, produces results. Just by seeing its classrooms you start to figure this out: the students are in matching uniforms, they chant and seem energized about learning, and, other than the chants, they’re orderly and respectful. To prove that KIPP works mathematically, up until now we’ve had to rely on pretty low-level analyses that show very high numbers of their students pass state accountability exams.

A new, rigorous analysis for the National Bureau of Economic Research changes that. Using a quasi-experimental research design that capitalizes on the large number of students applying to get into but ultimately rejected from one KIPP school in Lynn, Massachusetts, the researchers were able to compare students who entered the school with those who wanted to attend but were rejected due to space restrictions. This design helps the researchers isolate KIPP attendance from motivation, parental education, environmental factors, or any other variable that might be difficult to detect.

After these controls, KIPP attendees gained .35 of a standard deviation every year in math and .12 standard deviations each year in English. Results were even more positive for Limited English Proficiency and special education students (the demographics of KIPP Lynn lottery winners matched lottery losers and the district as a whole).

These demographic factors are important. KIPP is often accused of “skimming” the most talented or motivated public school students. Under that theory, which the paper debunks, KIPP could have a high percentage of low-income and minority students but attract the best of these groups. The research design of this paper allows the authors to conclude that’s not happening, at least in KIPP Lynn.

KIPP is also accused, based on the findings of a report on its San Francisco schools, of looking better than it is because of attrition. If only the best students stay in their schools (KIPP requires long hours and extremely hard work of its students), KIPP’s schools would look progressively better as fewer and fewer, more talented students survive. KIPP Lynn does not have this problem: Its students change schools no more or less often than lottery losers.

This is only one study, in one distinct location, and with only about 400 students, but it still merits attention. Most importantly, it is by far the most rigorous of all the evaluations thus far that specifically focus on KIPP. Journalistic stories and simplistic glances at the data are just not the same as comparing a treatment and control group over a number of years. Also, because KIPP is a national brand and its schools are relatively similar, this study is looking at more than just KIPP Lynn. The study will not help us figure out the difficult challenges involved in scaling up KIPP or similar programs, but it does answer many of the questions about their impact.