Joel Klein is a hoot. Klein, who served as Chancellor of the New York City Public Schools from 2002 to 2010, recently took to the opinion pages of The Washington Post to crown his friends and cronies the champions of education reform. Several alumni from the New York City Department of Education who presumably learned how to promote reform under Klein’s direction have assumed prominent leadership positions: John White is the superintendent in New Orleans, Cami Anderson in Newark, Jean-Claude Brizard in Chicago, Andres Alonso in Baltimore, and Marcia Lyles in Delaware’s Christina School District; similarly, Chris Cerf is the state commissioner of education in New Jersey. These names join others around the country, many trained by the Broad Superintendents Academy.

Klein argues that what sets these reformers apart from apologists for the failed status quo is that they have higher expectations. “Sure, educating children from difficult circumstances is often much harder,” he writes, “but the notion that schools can get much better results with those same kids than they’re now generally getting is no longer a matter of abstract debate. It’s now established fact.” The rhetoric here is, to my ear, quite remarkable. It doesn’t place children and their development in the foreground; rather, the emphasis is on what schools can extract from kids. It’s a dangerous rhetorical twist, because it points to accountability systems that can decouple school-performance measures from student learning. There’s little persuasive evidence that such systems can promote lasting achievement. Just last month, for example, the Board on Testing and Assessment of the National Research Council reported that test-based incentive programs have not had consistent and substantial effects on student achievement.

To support his claim, Klein points to a recent in-house study produced by KIPP, a chain of charter schools, showing that graduates from KIPP’s first two schools in New York City and Houston, overwhelmingly poor children of color, achieved four-year college graduation rates equivalent to those of white students. KIPP may indeed be doing great things, but any reputable social scientist would caution against concluding that participating in KIPP produced favorable outcomes without taking into account the mechanisms by which children are selected into and out of the program and its schools.

More peculiar is the use of KIPP as an example of the power of the school-reform movement. Joel Klein was Chancellor of the New York City Public Schools for eight years, and the individuals I named above all served as key deputies during his tenure. To the best of my knowledge, they had nothing to do with the performance of students attending these two KIPP schools.

On the other hand, Klein and his deputies presumably had a great deal to do with the performance of the 1.1 million students in the New York City public schools. So perhaps that’s a better place to look for the effect on the achievement gap of this concentration of school-reform talent.  What’s the evidence?

(Data are from the NAEP Data Explorer on the National Center for Education Statistics website)

Here are the facts: On the National Assessment of Educational Progress (NAEP), a highly respected measure of student achievement, fourth- and eighth-graders achieved modest growth in reading and math achievement from 2003 to 2009, which was “prime time” for the reforms that Klein and his team enacted. That growth is to be applauded; scores on NAEP do not rise by themselves. There is, however, no evidence that the gains observed in New York City outpaced what was observed over the same time-frame in other urban school districts that were notled by the current wave of reformers. In other words, NAEP scores in reading and math rose just as much in districts led by “apologists for the failed status quo” as they did in the district with the greatest concentration of “reform talent” in the nation.  Head-to-head with Atlanta, Boston, Charlotte, Chicago, Cleveland, the District of Columbia, Houston, Los Angeles and San Diego, in fourth- and eighth-grade reading and math,New York City gained more on NAEP between 2003 and 2009 11% of the time; gained less on NAEP 28% of the time; and was statistically indistinguishable from the comparison city 61% of the time. This is not strong evidence that the package of reforms promoted by this new cadre of school leaders in New York City resulted in better outcomes for children than the reforms pursued in other districts.

Moreover, the so-called “achievement gap”—group differences in educational achievement outcomes between white students, on the one hand, and black and Latino students, on the other—remained substantial at the conclusion of Joel Klein’s tenure, and didn’t budge between 2003 and 2009. In both reading and math, and in both fourth- and eighth-grade, the black-white and Latino-white differences in average performance on the NAEP among New York City students remains between .72 and .90 standard deviations, roughly the difference between the 50th percentile and the 80th percentile of overall student performance. The black-white and Latino-white group differences in New York City on the 2009 NAEP fourth- and eighth-grade science tests are even larger.

Does pointing these facts out mean that I think poor and minority children can’t learn? Does it make me an apologist for the failed status quo? I don’t think so. What it does mean is that I’m skeptical of unproven reforms that are championed by politicians, business leaders and philanthropists who seek to impose their view of what works on public education in the absence of credible evidence. Joel Klein contends that “the status quo is broken and that incremental change won’t work.” He and like-minded school leaders such as Michelle Rhee use these claims as a license to do just about anything that strikes their fancy, without regard to the time it takes to build the structures and cultures that can support reform.  The consequence is an unstable system that threatens to collapse at any moment. Substituting a sense of urgency for a deliberate and well-planned approach to incremental change may feel good, but it’s bad public policy.