Contact:
Catherine Durso, (303) 871-3598, cdurso@cs.du.edu
William J. Mathis, (802) 383-0058, wmathis@sover.net
URL for this press release: http://tinyurl.com/78n3yv5
BOULDER, CO (June 20, 2011) – In its second attempt to rank Los Angeles teachers based on “value-added” assessments derived from students’ standardized test scores, the Los Angeles Times has still produced unreliable information that cannot be used for the purpose the newspaper intends, according to new research released today by the National Education Policy Center, housed at the University of Colorado Boulder
Dr. Catherine Durso of the University of Denver studied the newspaper’s 2011 rankings of teachers and found that they rely on data yielding results that are unstable from year to year. Additionally, Durso found that the value-added assessment model used by the Times can easily impute to teachers effects that may in fact result from outside factors, such as a student’s poverty level or the neighborhood in which he or she lives.
“The effect estimate for each teacher cannot be taken at face value,” Durso writes. Instead, each teacher’s effect estimate includes a large “error band” that reflects the probable range of scores for a teacher under the assessment system.
“The error band . . . for many teachers is larger than the entire range of scores from the ‘less effective’ to ‘more effective’ designations provided by the LA Times,” Durso writes. As a consequence, the so-called teacher-linked effect for individual teachers “is also unstable over time,” she continues.
Durso found in particular that when teachers change schools, their rankings under the value-added assessments are likely to change.
An NEPC review in 2011 of the methodology that the Times relied on to produce the newspaper’s 2010 report ranking teachers based on “value-added assessment” similarly found those methods too flawed to produce reliable information. That report, Due Diligence and the Evaluation of Teachers, by Derek C. Briggs and Ben Domingue, can be found at: http://nepc.colorado.edu/publication/due-diligence.
Durso did find some small degree of improvements over the past year in the methodology used by the Times and by the social scientist with whom the newspaper contracted with to produce the value-added assessments of teachers, Richard Buddin. But those changes were not sufficient to make the database itself reliable in its rankings of teachers.
These failings have rendered the Times’ rankings not merely useless, but potentially harmful, according to Alex Molnar, NEPC’s publications director and a research professor at the University of Colorado Boulder.
“The Los Angeles Times has added no value to the discussion of how best to identify and retain the highest-quality teachers for our nation’s children,” Molnar says. “Indeed, it has made things worse. Based on this flawed use of data, parents are enticed into thinking their children’s teachers are either wonderful or terrible.”
“The Los Angeles Times editors and reporters either knew or should have known that their reporting was based on a social science tool that cannot validly or reliably do what they set out to quantify,” Molnar said. “Yet in their ignorance or arrogance they used it anyway, to the detriment of children, teachers, and parents.”
An Analysis of the Use and Validity of the Test-Based Teacher Evaluations Reported by the Los Angeles Times: 2011, by Catherine Durso,can be found on the web at: http://nepc.colorado.edu/publication/analysis-la-times-2011.
The National Education Policy Center unites a diverse group of interdisciplinary scholars from across the United States. The Center is guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence. To learn more about NEPC, please visit http://nepc.colorado.edu/.
This NEPC research brief was made possible in part by the support of the Great Lakes Center for Education Research and Practice. This report is also found on the GLC website at http://www.greatlakescenter.org/.