Report on teacher value-added impact improperly
ignores information contradicting its findings
Contact:
William J. Mathis, (802) 383-0058, wmathis@sover.net
Moshe Adler, (917) 453-4921, ma820@columbia.edu
URL for this press release: http://tinyurl.com/kpwt6ft
BOULDER, CO (April 10, 2014) – A highly influential but non-peer-reviewed report on teacher impact suffers from a series of errors in methodology and calculations, according to a new review published today.
Professor Moshe Adler reviewed two recent reports released in September 2013 as National Bureau of Economic Research working papers. Dr. Adler’s review for the Think Twice think tank review project is published today by the National Education Policy Center, housed at the University of Colorado Boulder School of Education.
Adler is an economist affiliated with both Columbia’s Urban Planning Department as well as the Harry Van Arsdale Jr. Center for Labor Studies at Empire State College, SUNY, and the author of Economics for the Rest of Us: Debunking the Science That Makes Life Dismal (New Press, 2010).
Adler reviewed Measuring the Impact of Teachers, parts I and II, written by Raj Chetty, John N. Friedman, and Jonah E. Rockoff. Part I is subtitled, Evaluating Bias in Teacher Value-Added Estimates, and Part II,Teacher Value-Added and Student Outcomes in Adulthood. Taken together, the two-part report asserts that students whose teachers have higher value-added scores achieve greater economic success later in life.
The documents (as is standard for NBER working papers) were not peer-reviewed, yet as Adler points out, the research on which they were based has gained extraordinary attention – turning up as references in President Obama’s 2012 State of the Union address, in expert court testimony by the principal author (Chetty), in extensive news coverage, and even as a justification for Chetty’s 2012 MacArthur Foundation “genius” award.
That sort of credibility, Adler suggests in his review, may not be warranted – as demonstrated by a series of problems that he finds with the new two-part report and the research that undergirds it.
The report’s own results reveal that calculating teacher value-added is unreliable, Adler writes. Additionally, the report includes a result that contradicts the central claim; it relies on an erroneous calculation to support a favorable result; and it assumes that the miscalculated result holds across students’ lifetimes – “despite the authors’ own research indicating otherwise,” the reviewer notes.
Finally, Adler explains, the studies relied on by the report as support for its methodology don’t actually provide that support.
“Despite widespread references to this study in policy circles, the shortcomings and shaky extrapolations make this report misleading and unreliable for determining educational policy,” Adler concludes.
Find Moshe Adler’s review on the NEPC website at:
http://nepc.colorado.edu/thinktank/review-measuring-impact-of-teachers.
Find Measuring the Impacts of Teachers, by Raj Chetty, John N. Friedman, and Jonah E. Rockoff, on the web at:
http://www.nber.org/papers/w19423 and http://www.nber.org/papers/w19424.
The Think Twice think tank review project (http://thinktankreview.org) of the National Education Policy Center (NEPC) provides the public, policymakers, and the press with timely, academically sound reviews of selected publications. NEPC is housed at the University of Colorado Boulder School of Education. The Think Twice think tank review project is made possible in part by support provided by the Great Lakes Center for Education Research and Practice.
The mission of the National Education Policy Center is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence. For more information on the NEPC, please visit http://nepc.colorado.edu/.
This review is also found on the GLC website at http://www.greatlakescenter.org/.