Review raises some technical concerns but praises report's contribution to the established research base showing minimal, and perhaps negative, average performance results

Contact: Gary Miron, (269) 599-7965; gary.miron@wmich.edu
Brooks Applegate, (269) 387-3886; brooks.applegate@wmich.edu
Kevin Welner, (303) 492-8370; kevin.welner@gmail.com

BOULDER, Colo. and TEMPE, Ariz. (June 24, 2009) -- A new report on the impact of charter schools on student performance finds that, on average, such schools perform no better than conventional public schools. The report, released last week, claimed to break new ground in its national coverage and in the detail of its analyses, and it received a great deal of attention from media and policy makers. A review of the report confirms its claims of a superior and extensive data base and finds that its analyses are largely sound, with some limitations that should have been shared in the report.

The report is Multiple Choice: Charter School Performance in 16 States, by the Center for Research on Education Outcomes (CREDO) at Stanford University. It was reviewed for the Think Tank Review Project by Gary Miron and Brooks Applegate, professors at Western Michigan University with extensive experience studying and evaluating charter school performance.

Multiple Choice draws its conclusions from the researchers' examination of longitudinal, student-level data compiled from 15 or 16 states (the reviewers note some lack of clarity on this point), covering 65-70% of the nation's charter schools. The report analyzes the achievement of students in these charter schools compared to that of matched students in traditional public schools.

The primary findings of the CREDO report show that charter school students' test performance is basically the same as the performance of students enrolled in traditional public schools. Because of their very large data base, the authors were able to tease out statistically significant differences in 54% of the charter schools studied, with the following results: "17 percent [of charters] provide superior education opportunities for their students. ... 37 percent, deliver learning results that are significantly worse than their student [sic] would have realized had they remained in traditional public schools."

In their review, Miron and Applegate summarize past research examining charter school performance and conclude that the CREDO findings are consistent with that earlier work. Charter schools, on average, appear to have achievement outcomes that are similar to, and perhaps worse, than traditional public schools. "The scope and relative rigor of the CREDO study reinforces the larger body of evidence which shows no overall impact of charter schools on performance," Miron and Applegate report.

The CREDO report also attempts to draw some state-level conclusions from their results, looking that three policies associated with more or less restrictiveness in the state charter laws: caps on the number of allowed charters in the state, restrictions on who can authorize the creation of a charter, and the allowance of appeals by charter applicants from a denial of authorization. The results were mixed, and the reviewers conclude that the analytic approach was undermined by the divergent manner in which these policies are implemented. The reviewers do, however, present their own secondary look at the state-level data (presented in their review) and uncover a pattern showing states doing better when they have fewer charters and when fewer of those charters are run by for-profit corporations.

Although too complex to be easily summarized, the review from Miron and Applegate also raises a series of technical questions regarding the report's analyses. Because of the potential value of the CREDO work, the reviewers urge the authors to answer those questions in technical follow-up papers to the report and in later work with their data base.

Find Gary Miron's and Brooks Applegate's review on the web at:
http://epicpolicy.org/thinktank/review-multiple-choice

CONTACT:
Gary Miron, Professor
Dept. of Educational Leadership, Research & Technology
College of Education, Western Michigan University
269-387-3883
gary.miron@wmich.edu

Kevin Welner, Professor and Director
Education and the Public Interest Center
University of Colorado at Boulder
(303) 492-8370
kevin.welner@gmail.com

About the Think Tank Review Project

The Think Tank Review Project (http://thinktankreview.org), a collaborative project of the ASU Education Policy Research Unit (EPRU) and CU-Boulder's Education and the Public Interest Center (EPIC), provides the public, policy makers, and the press with timely, academically sound reviews of selected think tank publications. The project is made possible by funding from the Great Lakes Center for Education Research and
Practice.

Kevin Welner, the project co-director, explains that the project is needed because, "despite their garnering of media attention and their influence with many policy makers, reports released by private think tanks vary tremendously in their quality. Many think tank reports are little more than ideological argumentation dressed up as research. Many others include flaws that would likely have been identified and addressed through the peer review process. We believe that the media, policy makers, and the public will greatly benefit from having qualified social scientists provide reviews of these documents in a timely fashion." He adds, "we don't consider our reviews to be the final word, nor is our goal to stop think tanks' contributions to a public dialogue. That dialogue is, in fact, what we value the most. The best ideas come about through rigorous critique and debate."

**********
###

The Education and the Public Interest Center (EPIC) at the University of Colorado at Boulder and the Education Policy Research Unit (EPRU) at Arizona State University collaborate to produce policy briefs and think tank reviews. Our goal is to promote well-informed democratic deliberation about education policy by providing academic as well as non-academic audiences with useful information and high quality analyses.

Visit EPIC and EPRU at http://www.educationanalysis.org/

EPIC and EPRU are members of the Education Policy Alliance
(http://educationpolicyalliance.org).

###
**********