The National Education Policy Center urges caution when reading the CREDO study of charter schools in New Orleans. Governor Bobby Jindal has already taken CREDO as evidence for the success of privatization.

NEPC says not so fast. In addition to technical issues in the study, the critics make the following observations:

“Even setting aside these concerns, the effect sizes reported for New Orleans—let alone for the state as a whole—are not impressive in terms of absolute magnitude. Differences of 0.12 standard deviations in reading and 0.14 in mathematics indicate that less than one half of one percent of the variation in test scores is explainable by membership in a charter school.

“The study’s methods raise concerns that the findings could easily be misinterpreted to inflate pro-charter conclusions. In context, there’s little to crow about: the results from Louisiana and New Orleans are not much different from the uninspiring national results; the results for the state’s suburban charter schools showed negative gain scores (somewhat less growth in charters than in the comparison schools); and the small positive results reported for New Orleans are confounded by the devastating aftermath of a unique disaster.”

An even more serious challenge to the study was posed by New Orleans-based “Research on Reforms,” which complained that the Louisiana Department of Education will not release student data to independent research organizations.

It wrote: “As long as the Louisiana Department of Education can determine to whom to release student records for research purposes, the reports produced thereof, such as the CREDO report, are nothing more than biased evaluations.”

“The Department of Education (DOE) maintains that it has the discretion to release de-identified student-level records to selected researchers, and that it has the discretion to deny the same student records to other researchers. And, for the past few years, that is what the DOE has done. CREDO received the student records, and, Research on Reforms, Inc., who submitted a public records request for the same student records, was denied. As long as the DOE gets to select its evaluators, i.e., its researchers, the impact of the state-takeover and the charter school movement will never be objectively evaluated.

“Specifically, the Department of Education (DOE) released de-identified student-level records to CREDO for the school years 2008-09, 2009-10, and 2010-11 and denied the student level records for the same school years to Research on Reforms, Inc. (ROR). Thus, ROR sued the DOE in October 2012 for violation of Louisiana’s Public Records Act. The matter is now in Civil District Court.”