Reviewers find Heartland Institute analysis simplistic, its data skewed and research support for its recommendations lacking

Contact: Edward G. Fierros, Villanova University, (610) 519-6969, Edward.fierros@villanova.edu
William Mathis, NEPC, (802) 383-0058, William.Mathis@colorado.edu

BOULDER, CO (December 14, 2010) – The 2010 State School Report Card: A state-by-state analysis of learning, efficiency, and standards, published by the Heartland Institute, purports to rank states on student achievement, education expenditures, and adherence to learning standards. In a review released today, Edward Fierros and Bridget Ann Rooney of Villanova University found that the Heartland report and its grades are very poorly grounded. The review cites little relevant research, provides no rationale to support its methods and fails to control for underlying variations from state to state in demographics and other characteristics.

The review, part of the Think Twice think tank review project, is published by the National Education Policy Center, housed at the University of Colorado at Boulder School of Education.

The Heartland report, authored by Herbert Walberg and Marc Oestreich,ranks the 50 states and the District of Columbia according to four indices:  (1) gains or losses in National Assessment of Education Progress math and reading test scores from 2005 to 2009; (2) an “index of outcomes” that calculates costs per graduate, per student, per “learning gain over time” and “per unit of learning between grades,” and the ratio of teachers to staff – all in the name of measuring “efficiency”; (3) “standards” – which combines a ranking concerning states’ inflation of proficiency claims and rankings (grades) assigned to each state by the Fordham Institute on the “rigor and content” of math, reading and science standards; and (4) a composite score derived from each state’s  first three indices.

Reviewers Fierros and Rooney explain that all three of the core indices are flawed – two of them seriously so. They note, for instance, that the report’s system of assigning letter grades based on the rankings is problematic because it forces roughly the same number of states into each letter grade category. This approach “guarantees that there will be failing states even though the numerical differences between the states are often not very large.” While many states have nearly identical scores, the methods used to calculate their final rank exaggerates their differences.

The Heartland report calls for expanded school choice options and the adoption of “Parent trigger laws” whereby parents whose children attend a “failing” school can force a charter school or other conversion. According to Fierros and Rooney, these recommendations aren’t based on the report card calculations, “but rather on several brief and uncritical references to past think tank reports.” At the same time they note the report ignores research literature that finds little or no benefit from such policy prescriptions.

While a study using national data to examine states’ education performance has the potential for sparking “a deep and careful examination” of what might account for differences in school performance among states, the Heartland Report Card is not that study, Fierros and Rooney conclude. Its methods skew state-level education outcomes, its analyses are simplistic, and it lacks evidence to support its advocacy for school choice policies as a remedy. “For these reasons,” Fierros and Rooney conclude, “the report is not useful in guiding policy or practice.”

Find Edward Fierros’s review on the NEPC website at:
http://nepc.colorado.edu/thinktank/review-2010-state-school-report

Find 2010 State School Report Card: A state-by-state analysis of learning, efficiency, and standards, by Herbert J. Walberg and Marc Oestreich, on the web at:
http://www.heartland.org/article/28630/2010_State_School_Report_Card.html.

The Think Twice think tank review project (http://thinktankreview.org), a project of the National Education Policy Center, provides the public, policy makers, and the press with timely, academically sound, reviews of selected think tank publications. The project is made possible in part by the generous support of the Great Lakes Center for Education Research and Practice.

The mission of the National Education Policy Center is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence.  For more information on NEPC, please visit http://nepc.colorado.edu/.

This review is also found on the GLC website at http://www.greatlakescenter.org/