Pro-voucher Polls Plagued by Biased Questions and Sampling Problems, Review Finds

Contact: Jon Lorence, (713) 743-3959; (email) jlorence@uh.edu
A. Gary Dworkin, (713) 743-3955; (email) gdworkin@uh.edu
Kevin Welner, (303) 492-8370; (email) kevin.welner@gmail.com

TEMPE, Ariz. and BOULDER, Colo. (February 13, 2009) -- A new report issued this week by the Friedman Foundation is part of a series of reports based on public opinion polls, each of which relies on polls containing poorly worded questions and suffering from problems in sampling. These problems were documented two months ago, in a study of the first 10 of these Friedman reports.

The earlier reports were reviewed for the Think Tank Review Project by professors Jon Lorence and A. Gary Dworkin, both of the University of Houston.

The 10 reports conclude that a majority of voters view public schools as performing unsatisfactorily, that respondents prefer private over public schools and want more educational choices, that vouchers should be available, and that potential voters are more likely to support candidates who back school choice proposals.

Lorence and Dworkin note that none of the reports cite any other surveys or research literature regarding opinions on vouchers or on public, private, or charter schools. By comparison, the reviewers cite a series of Gallup polls, conducted annually for Phi Delta Kappa, that include questions measuring public support for private-school vouchers and for public education in general. Those surveys have shown moderate fluctuation in their findings, but consistently find lesser support for voucher proposals than shown by the Friedman surveys. In fact, the Gallup surveys consistently show that more Americans oppose vouchers than support them.

The reviewers help readers to understand why the Friedman surveys might yield these more favorable results for school choice proposals. They identify two primary problems.

First, they note that that the population samples surveyed might not represent the voting populations of their states nor the population of parents for whose children school vouchers are intended. Information to establish sampling accuracy is largely missing from the reports, and in at least one of the three instances where sufficient data were provided, the low response rate raises a red flag about possible sample bias.

Additionally, Lorence and Dworkin found repeated instances in which the wording of questions appeared likely to bias the results. These questions appeared to be worded in ways to encourage respondents to favor private-school funding. The reviewers use the corresponding Gallup questions to highlight these problems with wording. Additionally, the survey questions in some instances used terms not widely understood, such as questions about proposals to grant tax credits to companies that finance private school scholarships (vouchers).

"Had the respondents been presented more neutral questions about tax credits and vouchers, the findings may have been less favorable towards these issues," the reviewers write.

Lorence and Dworkin observe that the 10 reports reviewed are aimed at driving a pro-school-choice agenda through the promotion of tax credits, vouchers, and charter schools. But in addition to problems with bias in the surveys themselves, they point out, there's a more fundamental problem: namely, that "while the responses of those surveyed in the 10 states may reflect their beliefs endorsing alternatives to public schools, their views should not be construed -- as they often are in these reports -- to mean that adopting vouchers and other pro-choice school policies will in fact improve the quality of education."

In short, Lorence and Dworkin conclude, the reports should not guide policy.

Find the review by Jon Lorence and A. Gary Dworkin on the web at:
http://epicpolicy.org/thinktank/review-voucher-surveys

Find the Friedman Foundation's Vermont report at:
http://www.friedmanfoundation.org/friedman/downloadFile.do?id=352

CONTACT:
Jon Lorence
University of Houston
(713) 743-3959
jlorence@uh.edu

A. Gary Dworkin
University of Houston
(713) 743-3955
gdworkin@uh.edu

Kevin Welner, Professor and Director
Education and the Public Interest Center
University of Colorado at Boulder
(303) 492-8370
kevin.welner@gmail.com

About the Think Tank Review Project

The Think Tank Review Project (http://thinktankreview.org), a collaborative project of the ASU Education Policy Research Unit (EPRU) and CU-Boulder's Education and the Public Interest Center (EPIC), provides the public, policy makers, and the press with timely, academically sound reviews of selected think tank publications. The project is made possible by funding from the Great Lakes Center for Education Research and
Practice.

Kevin Welner, the project co-director, explains that the project is needed because, "despite their garnering of media attention and their influence with many policy makers, reports released by private think tanks vary tremendously in their quality. Many think tank reports are little more than ideological argumentation dressed up as research. Many others include flaws that would likely have been identified and addressed through the peer review process. We believe that the media, policy makers, and the public will greatly benefit from having qualified social scientists provide reviews of these documents in a timely fashion." He adds, "we don't consider our reviews to be the final word, nor is our goal to stop think tanks' contributions to a public dialogue. That dialogue is, in fact, what we value the most. The best ideas come about through rigorous critique and debate."

**********
###

The Education and the Public Interest Center (EPIC) at the University of Colorado at Boulder and the Education Policy Research Unit (EPRU) at Arizona State University collaborate to produce policy briefs and think tank reviews. Our goal is to promote well-informed democratic deliberation about education policy by providing academic as well as non-academic audiences with useful information and high quality analyses.

Visit EPIC and EPRU at http://www.educationanalysis.org/

EPIC and EPRU are members of the Education Policy Alliance
(http://educationpolicyalliance.org).

###
**********