With Advertisements for the Common Core, the Editorial Board at The New York Times has offered its special brand of Kool-Aid journalism to the careless claim that 2013 NAEP data somehow prove education reform is a success:

The country is engaged in a fierce debate about two educational reforms that bear directly on the future of its schoolchildren: first, teacher evaluation systems that are taking hold just about everywhere, and, second, the Common Core learning standards that have been adopted by all but a few states and are supposed to move the schools toward a more challenging, writing-intensive curriculum.

Both reforms — or at least the principles behind them — got a welcome boost from reading and math scores released recently by the federal government. …

Two examples are the District of Columbia and Tennessee, among the first to install more ambitious standards and teacher evaluations. Tennessee jumped from 46th in the country in fourth-grade math two years ago to 37th, and from 41st in the nation to 34th in eighth-grade reading. The District of Columbia, though still performing below the national average, has also shown progress. The scores of its students improved significantly in both math and English.

Moreover, according to Education Secretary Arne Duncan, the eight states that managed to get the Common Core standards in place in time for the latest National Assessment of Educational Progress exams this year showed improvement from 2009 scores in either reading or math.

Kool-Aid journalism occurs when journalists relinquish their work as researchers and reporters to political appointees—in this case the Editorial Board of the NYT decides to turn Secretary Duncan’s baseless claims into statements of fact that support an editorial position. The Board concludes:

But the progress seen elsewhere — like Tennessee and the District of Columbia — shows that improvement is possible if the states strengthen their resolve and apply solutions that have been shown to work.

However, if the Editorial Board at the NYT had made even a basic effort at confirming Duncan’s claims, the Board could have discovered that NAEP data are complicated and cannot prove in any way that recent reforms are a success.

As I have detailed, and despite my not having any training as a journalist or as an investigative reporter, the Editorial Board could have benefitted from the following clarifications about NAEP that I found easily—all of which discredit Duncan’s claims and the Board’s position:

• From When You Hear Claims That Policies Are Working, Read The Fine Print by Matthew DiCarlo:

When I point out that raw changes in state proficiency rates or NAEP scores are not valid evidence that a policy or set of policies is “working,” I often get the following response: “Oh Matt, we can’t have a randomized trial or peer-reviewed article for everything. We have to make decisions and conclusions based on imperfect information sometimes.”

This statement is obviously true. In this case, however, it’s also a straw man. There’s a huge middle ground between the highest-quality research and the kind of speculation that often drives our education debate. I’m not saying we always need experiments or highly complex analyses to guide policy decisions (though, in general, these are always preferred and sometimes required). The point, rather, is that we shouldn’t draw conclusions based on evidence that doesn’t support those conclusions.

• Jim Horn offers a detailed three part series showing what NAEP does, and does not, reveal: Has Corporate Education “Reform” Earned Another 25 Years to Complete the Resegregation and Privatization Agendas? Part I, Part 2: Has Corporate Education “Reform” Earned Another 25 Years to Complete the Resegregation and Privatization Agendas?, and Part 3: Has Corporate Education “Reform” Earned Another 25 Years to Complete the Resegregation and Privatization Agendas?, in which Horn clarifies: “The continued hydroplaning of NAEP scores nationally allows the world to see what the corporate education losers will not admit: the 3rd generation of test based accountability is not working. “
• In On the latest D.C. NAEP Miracle, Gary Rubinstein directly confronts the claims of reform success:

This shows that the places with the greatest gains were D.C., Tennessee, and Indiana, three places that have embraced the corporate reform strategy of testing, closing down schools, and opening charters.  If this was the only data we had access to, it would seem to prove that “the ends justify the means” when it comes to education reform….

There are many other things to analyze, and I’m looking forward to reading how others analyze the data.  For example, it is curious that Louisiana had ‘gains’ that were smaller than the national average despite that state having, certainly, the most aggressive reforms occurring.  For ‘reformers’ who are so obsessed with test scores and test score gains, this is certainly something that shouldn’t be ignored.  Also, Washington and Hawaii were pretty high up on the ‘growth’ numbers even though Washington does not have charter schools and Hawaii has been very slow to adopt Race To The Top reforms so their ‘gains’ can’t be attributed to those.

I’m still pretty confident that in the long run education reform based primarily on putting pressure on teachers and shutting down schools for failing to live up to the PR of charter schools will not be good for kids or for the country, in general.  I hope politicians won’t accept the first ‘gains’ chart without putting it into context with the rest of the data.

• Latest NAEP Results, by G.F. Brandenburg exposes that DC gains pre-date the reforms championed by Duncan and the NYT:

First of all, the increases in some of the scores in DC (my home town) are a continuation of a trend that has been going on since about 2000. As a result of those increases, DC’s fourth grade math students, while still dead last in the nation, have nearly caught up with MISSISSIPPI, the lowest-scoring state in the US.

You will have to strain your imagination to see any huge differences between the trends pre-Rhee and post-Rhee. (She was installed after testing was over in 2007.)…

So, the Educational DEforms instituted by Rhee, Henderson, and their corporate masters have not produced the promised miracles.

•   Bruce Baker explains in A few quick thoughts and graphs on Mis-NAEP-ery:

Yesterday gave us the release of the 2013 NAEP results, which of course brings with it a bunch of ridiculous attempts to cast those results as supporting the reform-du-jour. Most specifically yesterday, the big media buzz was around the gains from 2011 to 2013 which were argued to show that Tennessee and Washington DC are huge outliers – modern miracles – and that because these two settings have placed significant emphasis on teacher evaluation policy – that current trends in teacher evaluation policy are working – that tougher evaluations are the answer to improving student outcomes – not money… not class size… none of that other stuff.

I won’t even get into all of the different things that might be picked up in a supposed swing of test scores at the state level over a 2 year period. Whether 2 year swings are substantive and important or not can certainly be debated (not really), but whether policy implementation can yield a shift in state average test scores in a two  year period is perhaps even more suspect….

Is Tennessee’s 2-year growth an anomaly? we’ll have to wait at least another two years to figure that out. Was it caused by teacher evaluation policies? That’s really unlikely, given that those states that are equally and even further above their expectations have approached teacher evaluation in very mixed ways and other states that had taken the reformy lead on teacher policies – Louisiana and Colorado – fall well below expectations.

• And finally, the NYT Editorial Board could consider some more careful journalism, such as When Bad Things Happen to Good NAEP Data by Stephen Sawchuk.

As it stands, the position taken by the NYT Editorial Board lacks even the barest qualities of credibility, but it does expose the utter failure of Kool-Aid journalism.