Examination of eight states uses sound methods and provides honest presentation

Contact: Derek Briggs, (303) 492-6320; derek.briggs@colorado.edu
Kevin Welner, kevin.welner@gmail.com

TEMPE, Ariz. and BOULDER, Colo. (May 27, 2009) -- A new report from RAND on charter schools in eight states generally finds them to have modest or insignificant effects on an array of outcomes, including student achievement, student attainment, integration, and public school competition. A review of the RAND report raises questions about all four of the report's sections, particularly stressing some weaknesses in the data and analyses regarding the integration and competition findings. But overall, the review concludes that the report is of high quality and makes an important contribution to the empirical literature on charter school effectiveness.

The report is Charter Schools in Eight States: Effects on Achievement, Attainment, Integration and Competition and was written for RAND by a team led by Ron Zimmer. It was reviewed for the Think Tank Review Project by Derek Briggs of the University of Colorado at Boulder, an expert on social science research methods.

The RAND report examines four areas: characteristics of students transferring to charter schools; effects of charters on test-score gains for students switching to them from public schools; effects of charter high school attendance on graduation and on enrolling in college; and competition effects from charter schools on students attending nearby public schools.

This new report comes as charter schools -- public schools that operate with greater independence from local school district authorities -- are getting fresh attention, with President Barack Obama urging states to allow more charters.

RAND's study found:
• Insignificant effects on reading and math achievement in five jurisdictions and small negative effects in two others.
• Positive effects for graduation rates and matriculation in college, but only for two jurisdictions for which there were data.
• No evidence that charter schools skim higher-achieving students from traditional public schools, and no evidence that charters lead to increased racial or ethnic stratification.
• No evidence of a "competition effect" leading to an improvement or decline in the scores of local traditional public school students when charters enter a particular educational marketplace.

"On the whole, the methods used in this report are exemplary," Briggs writes in his summary. "The authors describe their statistical analyses in a transparent manner that makes it possible for readers to form their own opinions about the strength of the argument being advanced."

Briggs does point out weaknesses that the data and analyses underlying the finding of no evidence that charter schools are skimming high-achieving students away from public schools or leading to increased racial/ethnic stratification. The study uses highly aggregated data, meaning that a great deal of stratification and skimming could exist but still not be seen by the approach used in the report. Individual schools can be segregated but when the data are collapsed (or "aggregated"), with all the school-level data mixed together in one large pot, the aggregated populations of students in charter schools and traditional public schools might look very similar.

On the issue of achievement, the most defensible charter effects estimated in this study are based on data from students who switch to charters in middle school or high school after having attended a traditional public school. Briggs notes that while this choice may strengthen the study's internal validity, it limits the study's external validity.

The Briggs review raises various other questions and concerns, making it a useful companion piece to the RAND report itself -- a report that Briggs praises as "sophisticated, thoughtful, and even-handed."

Find Derek Briggs' review on the web at:
http://epicpolicy.org/thinktank/review-Charter-Schools-Eight-States

CONTACT:
Derek C. Briggs
Chair and Associate Professor, Research and Evaluation Methodology Program
School of Education
University of Colorado at Boulder
(303) 492-6320
derek.briggs@colorado.edu

Kevin Welner
Director, Education and the Public Interest Center
University of Colorado at Boulder
kevin.welner@gmail.com

About the Think Tank Review Project

The Think Tank Review Project (http://thinktankreview.org), a collaborative project of the ASU Education Policy Research Unit (EPRU) and CU-Boulder's Education and the Public Interest Center (EPIC), provides the public, policy makers, and the press with timely, academically sound reviews of selected think tank publications. The project is made possible by funding from the Great Lakes Center for Education Research and Practice.

Kevin Welner, the project co-director, explains that the project is needed because, "despite their garnering of media attention and their influence with many policy makers, reports released by private think tanks vary tremendously in their quality. Many think tank reports are little more than ideological argumentation dressed up as research. Many others include flaws that would likely have been identified and addressed through the peer review process. We believe that the media, policy makers, and the public will greatly benefit from having qualified social scientists provide reviews of these documents in a timely fashion." He adds, "we don't consider our reviews to be the final word, nor is our goal to stop think tanks' contributions to a public dialogue. That dialogue is, in fact, what we value the most. The best ideas come about through rigorous critique and debate."

**********
###

The Education and the Public Interest Center (EPIC) at the University of Colorado at Boulder and the Education Policy Research Unit (EPRU) at Arizona State University collaborate to produce policy briefs and think tank reviews. Our goal is to promote well-informed democratic deliberation about education policy by providing academic as well as non-academic audiences with useful information and high quality analyses.

Visit EPIC and EPRU at http://www.educationanalysis.org/

EPIC and EPRU are members of the Education Policy Alliance
(http://educationpolicyalliance.org).

###
**********