Despite potential of new methods to measure the impact of
education advocacy groups, report using them comes up short

Contact: 
William J. Mathis, (802) 383-0058, wmathis@sover.net
Robin Rogers, (347) 989-7869, Robin.Rogers@qc.cuny.edu

URL for this press release: http://tinyurl.com/plojjao

BOULDER, CO (April 16, 2015) – A recent Brookings Institution report seeks to measure the impact of advocacy groups on education reform policy. A review of that report published today finds that while it uses research methods that might be useful, the report’s shortcomings dictate that neither those methods nor the report’s conclusions should be accepted without additional research.

Robin Rogers and Sara Goldrick-Rab reviewed the report, Measuring and Understanding Education Advocacy, for the Think Twice think tank review project. The review is published by the National Education Policy Center, housed at the University of Colorado Boulder School of Education.

Professor Rogers teaches Sociology at Queens College and the City University of New York Graduate Center. Her research focuses on politics, policy, and philanthropy. Professor Goldrick-Rab teaches Educational Policy Studies and Sociology at the University of Wisconsin-Madison, and is founding director of the Wisconsin HOPE Lab, the nation’s only translational research laboratory focused on college affordability.

Measuring and Understanding Education Advocacy, written for Brookings by Grover J. (Russ) Whitehurst and others, offers two newer tools for measuring the impact of advocacy groups on education reform policy: Surveys with Placebo (SwP), designed to measure more accurately the influence of advocacy groups, and Critical Path Analysis (CPA), to identify which tactics were successful in influencing reform. The report contends that coordination among advocacy groups strengthens their impact and that the perceived impact of advocacy groups tracks closely with policy outcomes.

Rogers and Goldrick-Rab conclude that the two methodologies might have potential for education policy research. They point out, however, that both “are more limited than the report acknowledges.”

The reviewers explain that “the research is a small case study of three states, with a low response rate for the SwP and CPA based on advocacy groups’ self-reported tactics.”

The report, they add, lacks enough information about responses to the Surveys with Placebo or about the selection of the advocacy groups. This undermines that ability of the report’s readers to adequately assess either usefulness of its methods or validity of its conclusions.

“Finally, there is not a strong connection between the evidence presented in the report and its conclusions,” Rogers and Goldrick-Rab conclude. “We therefore caution against adoption of the methods or reliance on the conclusions presented in this report without significant further research.”

Find the review by Robin Rogers and Sara Goldrick-Rab on the NEPC website at:
http://nepc.colorado.edu/thinktank/review-measuring-education-advocacy

Find Measuring and Understanding Education Advocacy, by Grover J. (Russ) Whitehurst and published by the Brookings Institution, on the web at:
http://www.brookings.edu/research/reports/2015/03/04-education-advocacy-whitehurst.

The Think Twice think tank review project (http://thinktankreview.org) of the National Education Policy Center (NEPC) provides the public, policymakers, and the press with timely, academically sound reviews of selected publications. NEPC is housed at the University of Colorado Boulder School of Education. The Think Twice think tank review project is made possible in part by support provided by the Great Lakes Center for Education Research and Practice.

The mission of the National Education Policy Center is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence.  For more information on the NEPC, please visit http://nepc.colorado.edu/.

This review is also found on the GLC website at http://www.greatlakescenter.org/