Education Trust approach to identifying targets for turnarounds has too many flaws to be useful

Contact: Jaekyung Lee, (716) 645-1132; JL224@buffalo.edu
Nikki Rashada McCord, EPIC, (303) 735-5290; Nikki.McCord@colorado.edu

BOULDER, Colo. and TEMPE, Ariz. (April 7, 2010) -- The Education Trust recently offered a methodology for identifying so-called "stuck schools," purported to be most in need of targeted education reform strategies. A Think Tank Review released today on the group's report, however, finds it relies on misleading data and unreliable methodology, making its policy application problematic.

Stuck Schools was written for The Education Trust by Natasha Ushomirsky and Daria Hall. Focusing on schools in Maryland and Indiana to demonstrate its methods, the report sorts schools by both status and growth. That is, the report includes school-level test score information focused on baseline achievement status and also achievement gains over five years. The authors then identify those that both don't perform well at the baseline and don't improve well over time--the so-called "Stuck Schools" of the report's title.

The report comes as the Obama Administration is shifting the emphasis in the No Child Left Behind act to identify the lowest-performing five percent of schools in each state, based on student academic achievement, student growth, and graduation rates.

In his review of the report for the Think Tank Review Project, Professor Jaekyung Lee of the SUNY Buffalo finds numerous flaws. Although the authors never say so explicitly, they assume that schools improve in a straight line over time. That assumption, however, misses other patterns of growth, as Lee demonstrates by presenting the data from a Maryland school whose actual growth pattern is clearly not linear and that doesn't necessarily warrant its "stuck school" label.

Lee also points out that some of the report's key results may be due to a norm-referenced methodology that guarantees "failed" schools independent of any true performance or improvement level by the school, as well as a well-known statistical artifact called "regression to the mean"--such that school improvement or decline would not be attributable to anything more than measurement error. And he questions the assumption that underlies the report's title and the schools' label as "stuck." That, Lee observes, "suggests that the schools themselves--rather than the structural and resource issues within which those schools carry on--should be blamed and 'turned around.'"

Summing up his review, Lee writes: "The overall conceptual framework of the report helps bring more attention to the issues of validating and using school-level performance trend data for accountability." But he warns that the "report's methods are so simplistic, arbitrary and poorly fitting to the report's own assumptions that it is more harmful to sound policymaking than helpful."

The Think Tank Review Project (http://thinktankreview.org), a collaborative project of University of Colorado at Boulder's Education and the Public Interest Center (EPIC) and the ASU Education Policy Research Unit (EPRU), provides the public, policy makers, and the press with timely, academically sound reviews of selected think tank publications. The project is made possible by funding from the Great Lakes Center for Education Research and Practice.

Find Jaekyung Lee's review on the web at:
http://epicpolicy.org/thinktank/review-stuck-schools

Find Stuck Schools by Natasha Ushomirsky and Daria Hall on the web at:
http://www.ecs.org/html/Document.asp?chouseid=8440

CONTACT:
Jaekyung Lee
University at Buffalo-State University of New York
(716) 645-1132
JL224@buffalo.edu

Nikki Rashada McCord
Associate Director
Education & the Public Interest Center (EPIC)
University of Colorado at Boulder
(303) 735-5290
Nikki.McCord@colorado.edu

The Education and the Public Interest Center (EPIC) at the University of Colorado at Boulder and the Education Policy Research Unit (EPRU) at Arizona State University collaborate to produce policy briefs and think tank reviews. Our goal is to promote well-informed democratic deliberation about education policy by providing academic as well as non-academic audiences with useful information and high quality analyses.

Visit EPIC and EPRU at http://www.educationanalysis.org/

EPIC and EPRU are members of the Education Policy Alliance
(http://educationpolicyalliance.org).

###
**********