Research uses bad methodology, weak data and incorporates mistakes of earlier work, reviewer finds
Contact: Joydeep Roy, (202) 331-5526; jroy@epi.org
Kevin Welner, (303) 492-8370; kevin.welner@colorado.edu
BOULDER, Colo. and TEMPE, Ariz. (January 14, 2010) -- A recent report contends that providing private school vouchers would create job opportunities in five poor, rural South Carolina counties. A new review of that report, however, finds that it is built on seriously flawed assumptions and thus offers little insight into the effects of school vouchers.
The report, How School Choice Can Create Jobs for South Carolina, authored by Sven Larson, is published by the South Carolina Policy Council Education Foundation. It was reviewed for the Think Tank Review Project by Joydeep Roy of Georgetown University and the Economic Policy Institute in Washington, D.C.
How School Choice Can Create Jobs for South Carolina adopts the assumptions and analyses set forth in an earlier study. Both studies assert that communities where school vouchers are offered display higher rates of entrepreneurship and self-employment. The argument is apparently that vouchers are linked to individual freedom for those involved -- in contrast to the non-freedom associated with government-run schools -- and that the decrease in government involvement and increase in competition for students is supposed to spur competition and innovation, which is linked to self-employment and entrepreneurship.
Extrapolating data from the earlier study, the new report purports to precisely calculate the number of firms and jobs that would be created with a voucher program in place in the five South Carolina counties. Yet as Roy points out in his review, the original study suffers from a series of methodological flaws. Most importantly, the study is based on weak data that simply cannot confirm any kind of causal relationship between vouchers and entrepreneurial status.
Roy also notes that the entire argument in the earlier study was based on voucher programs in place by 1990, but the study never identifies the programs. In fact, the only programs that appear to have existed at that time were in Milwaukee voucher program (begun in 1990) and in Vermont and Maine, which had long had quirky tuitioning programs allowing remote, rural students to attend private, non-religious schools. "Lessons drawn from these unique programs and circumstances may have limited applicability to other areas and programs," Roy observes.
The Larson report uncritically applies the results of that earlier, flawed research, Roy writes. The new report then adds another unproven set of assumptions. Most importantly, it assumes without justification that there exists a significant "multiplier effect" in jobs created that are spun off of entrepreneurial activities. That is, it "assumes that small businesses would expand by 25 percent due to voucher programs, and that these will employ people at the same rate as existing small businesses of the same size," Roy explains. But, "new small businesses are unlikely to generate the same employment prospects as existing, older ones, since the latter have -- by definition -- been in the business for a while."
So the report comes up with an essentially speculative number representing increased self-employment, decides that this is the equivalent of entrepreneurship and that these new entrepreneurs are starting small businesses that mirror existing South Carolina small businesses in terms of their size and employment.
The upshot, says Roy, is a paper that relies more on rhetoric and less on authentic research. "The rhetoric of the report and the approaches used suggest a clear goal of championing a positive effect of voucher programs, with little or no concern for providing a careful analysis of the question at hand," Roy concludes. With inadequate methodology and "significant biases and omissions" undermining any potential for valid findings, it offers "little, if any, help in guiding policymakers, educators or the public."
Find Joydeep Roy's review of How School Choice Can Create Jobs for South Carolina on the web at http://epicpolicy.org/thinktank/review-How-School-Choice.
Find Sven Larson's report, How School Choice Can Create Jobs for South Carolina, published by the South Carolina Policy Council Education Foundation at http://scpolicycouncil.com/images/SchoolChoice.pdf
CONTACT:
Joydeep Roy
Economic Policy Institute & Georgetown University
(202) 331-5526
jroy@epi.org
Kevin Welner, Professor and Director
Education and the Public Interest Center
University of Colorado at Boulder
(303) 492-8370
kevin.welner@colorado.edu
About the Think Tank Review Project
The Think Tank Review Project (http://thinktankreview.org), a collaborative project of the ASU Education Policy Research Unit (EPRU) and CU-Boulder's Education and the Public Interest Center (EPIC), provides the public, policy makers, and the press with timely, academically sound reviews of selected think tank publications. The project is made possible by funding from the Great Lakes Center for Education Research and Practice.
Kevin Welner, the project co-director, explains that the project is needed because, "despite their garnering of media attention and their influence with many policy makers, reports released by private think tanks vary tremendously in their quality. Many think tank reports are little more than ideological argumentation dressed up as research. Many others include flaws that would likely have been identified and addressed through the peer review process. We believe that the media, policy makers, and the public will greatly benefit from having qualified social scientists provide reviews of these documents in a timely fashion." He adds, "we don't consider our reviews to be the final word, nor is our goal to stop think tanks' contributions to a public dialogue. That dialogue is, in fact, what we value the most. The best ideas come about through rigorous critique and debate."
**********
###
The Education and the Public Interest Center (EPIC) at the University of Colorado at Boulder and the Education Policy Research Unit (EPRU) at Arizona State University collaborate to produce policy briefs and think tank reviews. Our goal is to promote well-informed democratic deliberation about education policy by providing academic as well as non-academic audiences with useful information and high quality analyses.
Visit EPIC and EPRU at http://www.educationanalysis.org/
EPIC and EPRU are members of the Education Policy Alliance
(http://educationpolicyalliance.org).
###
**********