Leading expert finds Fordham Foundation study claiming low-performing students gain at the expense of high-achievers uses faulty methods and reaches a faulty conclusion.

Contact:  

Jamie Horwitz
202-549-4921
jhdcpr@starpower.net

Jaekyung Lee 
716-645-1132
JL224@buffalo.edu
 

BOULDER, CO-– Is it true that education policies are so skewed toward boosting academic achievement among low-performing students that high-performing students suffer? A recent report jointly released by the Fordham Institute and Northwest Evaluation Association, Do High Flyers Maintain Their Altitude? argues that the academic performance of high-achieving students is being undermined by a policy focus on lower-achieving students. Is this claim supported by evidence?

The National Education Policy Center (NEPC) asked University at Buffalo, SUNY, professor Jaekyung Lee to review the High Flyers report as part of NEPC’s Think Twice think tank review project. Lee, a nationally known expert on accountability and equity issues in education, found that the report’s conclusions rested on biased methodology and misleading arguments.

The review is published by the National Education Policy Center, housed at the University of Colorado Boulder School of Education.

Do High Flyers Maintain Their Altitude? tracks reading and math achievement trends for students who scored extremely well on the Measures of Academic Progress, and concludes that while three in five such students maintain high-achiever status, substantial numbers fall behind compared with their original performance, ultimately costing them access to such benefits as greater college choice and greater merit-based financial aid. It suggests that a possible reason for some high achievers slowing down is the focus on low-performing schools and students, prompted by the federal No Child Left Behind act, or on struggling readers on the part of the Reading First literacy strategy.

Lee concludes that the report offers little of value to policymakers. Indeed, he notes, “the report’s arguments about the loss of potential human capital and about a purported trade-off between excellence and equity can be more harmful than helpful.”

In his review, Lee describes a number of flaws in the report, including “its black-box approach that assumes a link between its findings and NCLB-related policies.” Lee also points to two key elements of the report that are unclear and inconsistent: the study’s approach to defining high achievers and its tracking of academic progress over time. At one point, the report uses percentile ranks in each grade, which is guaranteed to produce “winners” and “losers,” and elsewhere it uses developmental scale scores, which as Lee observes “allow for continuing growth regardless of change in relative status; thus  all students can be winners or losers.” As Lee points out, it is difficult for readers to make sense of these decisions, because the report fails to provide complete technical information on its methods, such as the construction of the samples of public school students on which the report’s findings are based.

To test the High Flyers report’s findings, Lee constructed a similar study of his own using a different national dataset, and he found results varied widely depending on which of the report’s two metrics was used. Lee also found that the study didn’t adequately account for regression to the mean, a statistical artifact that occurs when researchers examine the difference between two imperfectly correlated measures—in this case, achievement tests taken by students at different grades. Due to this measurement phenomenon, “lower-performing students tend to improve their performance status more than higher-performing students,” Lee notes. This occurs even if there is no actual change in the achievement being measured.
 

Find Jaekyung Lee’s review on the NEPC website at:
http://nepc.colorado.edu/thinktank/review-high-flyers

Find Do High Flyers Maintain Their Altitude? co-authored by Yun Xiang, Michael Dahlin, John Cronin, Robert Theaker, and Sarah Durant,on the web at:
http://www.edexcellence.net/publications-issues/publications/high-flyers.html       

The Think Twice think tank review project (http://thinktankreview.org), a project of the National Education Policy Center, provides the public, policy makers, and the press with timely, academically sound, reviews of selected publications. The project is made possible in part by the support of the Great Lakes Center for Education Research and Practice (GLC).

The mission of the National Education Policy Center, housed at the University of Colorado Boulder School of Education,is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. NEPC is guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence.  For more information on NEPC, please visit http://nepc.colorado.edu/

This review is also found on the GLC website at http://www.greatlakescenter.org/