Reviewer explains that study’s own data contradict authors’ conclusions
Contact:
William J. Mathis, (802) 383-0058, wmathis@sover.net
Francesca López, (520) 621-0307, falopez@email.arizona.edu
URL for this press release: http://tinyurl.com/mqyft8l
BOULDER, CO (September 30, 2014) – A recent meta-analysis of charter-school effects overstates its own findings, according to a new review published today.
Francesca López, an education professor at the University of Arizona, reviewed A Meta-Analysis of the Literature on the Effect of Charter Schools on Student Achievement for the Think Twice think tank review project. The review is published by the National Education Policy Center, housed at the University of Colorado Boulder School of Education.
The report was published in August by the Center on Reinventing Public Education at the University of Washington. The report, by Julian R. Betts and Y. Emily Tang, draws on data from 52 studies to conclude that charters benefited students, particularly in math.
“This conclusion is overstated,” writes López in her review. The actual results, she points out, were not positive in reading, not significant for high school math, and yielded only very small effect sizes for elementary and middle school math.
The reviewer also explains that the authors wrongly equate studies of students chosen for charter schools in a lottery with studies that rely on random assignment. Because schools that use lotteries do so because they’re particularly popular, those studies aren’t appropriate for making broad comparisons between charter and traditional public schools, López writes.
The review identifies other flaws as well, including the report’s assertion of a positive trend in the effects of charter schools, even though the data show no change in those effects; its exaggeration of the magnitude of some effects; and its claim of positive effects even when they are not statistically significant. Taken together, she says, those flaws “render the report of little value for informing policy and practice.”
“The report does a solid job describing the methodological limitations of the studies reviewed, then seemingly forgets those limits in the analysis,” López concludes.
Find Francesca López’s review on the NEPC website at:
http://nepc.colorado.edu/thinktank/review-meta-analysis-effect-charter.
Find A Meta-Analysis of the Literature on the Effect of Charter Schools on Student Achievement, by Julian R. Betts and Y. Emily Tang and published by the Center on Reinventing Public Education, on the web at:
http://www.crpe.org/publications/meta-analysis-literature-effect-charter-schools-student-achievement.
The Think Twice think tank review project (http://thinktankreview.org) of the National Education Policy Center (NEPC) provides the public, policymakers, and the press with timely, academically sound reviews of selected publications. NEPC is housed at the University of Colorado Boulder School of Education. The Think Twice think tank review project is made possible in part by support provided by the Great Lakes Center for Education Research and Practice.
The mission of the National Education Policy Center is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence. For more information on the NEPC, please visit http://nepc.colorado.edu/.
This review is also found on the GLC website at http://www.greatlakescenter.org/