The authors of this report created “efficiency scores” for Milwaukee schools and, on the basis of those scores, draw conclusions about the relative efficiency of Milwaukee’s charter and traditional public schools. A first set of scores was generated by dividing test scores in math and science by per-pupil funding. In a second analysis, “efficiency scores” were estimated for school types using a regression model that also accounted for selected student demographic characteristics. Several major problems arise from the authors’ approach: First, test scores do not comprehensively represent the purposes of schools. Second, threats to the validity of the author’s assumption that there is uniform financial accounting across schools and types are not addressed. Third, the analytic description of the study was incomplete, making interpretation difficult. Fourth, the report makes strong but unmeasured claims about the superior “efficiency” of charter schools based on their having greater autonomy. Finally, the authors did not appear to adjust for selection effects – effects that would prove fatal to their conclusions.  As a result of its manifold flaws, and because its conclusions are not supported by the evidence presented, the report is of little if any use to policymakers.

Update: Will Flanders and CJ Szafir, the report's authors, have posted a response to the review. The response can be found immediately below Cobb's original review of their report and at: http://www.will-law.org/will-blog-will-responds-criticism-colorado-group/.

Casey Cobb's rejoinder to the authors' response to his review is posted immediately below the authors' response.