Critique of teacher evaluation practices is sound and offers good recommendations, but review notes failure to draw upon promising strategies already in use
Contact: Raymond L. Pecheone, (650) 723-4106; (email) pecheone@stanford.edu
Kevin Welner, (303) 492-8370; (email) kevin.welner@gmail.com
TEMPE, Ariz. and BOULDER, Colo. (Aug. 5, 2009) -- A recent report from the New Teacher Project, based on surveys concerning teacher evaluations systems in 12 school districts, describes serious problems. It urges changes to establish more effective systems that will be able to discern the highest (and lowest) quality teachers. A new review of that report finds it well grounded and praises the report's policy recommendations as sound and consistent with prior research. The review does, however, note that the report lacks important details about its own methods and fails to consider or to draw upon any promising teacher evaluation strategies in current use.
The report is The Widget Effect: Our National Failure to Acknowledge and Act on Teacher Differences. It was written by Daniel Weisberg, Susan Sexton, Jennifer Mulhern, and David Keeling for The New Teacher Project, a non-profit consulting firm that focuses on teacher quality. It was reviewed for the Think Tank Review Project by Raymond Pecheone, Co-Executive Director of the Stanford University School Redesign Network and Director of the Performance Assessment for California Teachers, and Ruth Chung Wei, Director of Assessment Research and Development at the Stanford University School Redesign Network.
The report and the reviewers agree that extensive research points to the importance of teacher effectiveness in boosting student achievement, and thus the importance of accurately evaluating teachers for their performance in the classroom.
The review gives credit to The Widget Effect's authors for what they present: an extensive analysis of teacher evaluation systems in 12 districts across four states. The report is based on surveys of current and former teachers as well as administrators; on an examination of teacher evaluation records in the districts surveyed; and on interviews with school district leaders, school board officials, labor relations and human resources experts, and union leaders. Its extensive reporting of the data collected enables the authors "to build a compelling case for the inadequacy of the current teacher evaluation processes, uses, and policies in the four states," Pecheone and Wei write.
The reviewers praise the report's overall quality: "The picture the report paints of the landscape of current practices in teacher evaluation is an indictment of a broken system that is perpetuated by a culture that refuses to recognize and deal with incompetence. The report's careful march through the data collected by the authors leads to findings that are consistent with conclusions drawn by other reports about the current state of affairs in teacher evaluation."
The report offers four primary recommendations for teacher evaluation, advocating systems that fairly and accurately differentiate between teachers' effectiveness, have trained evaluators who are held accountable, use evaluations to inform key decisions, and have dismissal policies that offer due process and provide lower-stakes options for ineffective teachers to exit. Pecheone and Wei concluded that these "appear to represent reasonable strategies for improving a broken teacher evaluation system."
The report falls short, however, in its omission of key details that might help readers to understand the sampling choices. The reviewers write: "it is unclear ... how and why particular districts were selected, and whether they represent the range of teacher evaluation practices being implemented in school districts and states across the United States." Omissions in the report's description of its methodology (e.g., sampling strategy and survey response rates) and its sample lead to questions about the generalizability of the findings.
Pecheone and Wei also point out that the report does not consider important prior research in two areas: (1) why many evaluation systems don't work well (for example, because they rely on administrators with limited time and pedagogical expertise, tools and processes that have a limited relationship with effective teaching); and (2) how some innovative and effective evaluation systems do work.
The latter omission is particularly important, because considering these evaluation systems and research about those systems would have allowed the authors to draw upon current successes in designing their recommendations.
Transforming the system rather than tinkering around the edges, Pecheone and Wei conclude, will require broader thinking and a commitment to provide much greater investments and support for innovation to build, test, and audit evaluation systems that can stand up to public scrutiny and be practically feasible.
Find the review by Raymond Pechone and Ruth Chung Wei on the web at:
http://epicpolicy.org/thinktank/review-Widget-Effect
CONTACT:
Raymond L. Pecheone, Co-Executive Director
Stanford University School Redesign Network
(650) 723-4106
pecheone@stanford.edu
Kevin Welner, Professor and Director
Education and the Public Interest Center
University of Colorado at Boulder
(303) 492-8370
kevin.welner@gmail.com
About the Think Tank Review Project
The Think Tank Review Project (http://thinktankreview.org), a collaborative project of the ASU Education Policy Research Unit (EPRU) and CU-Boulder's Education and the Public Interest Center (EPIC), provides the public, policy makers, and the press with timely, academically sound reviews of selected think tank publications. The project is made possible by funding from the Great Lakes Center for Education Research and Practice.
Kevin Welner, the project co-director, explains that the project is needed because, "despite their garnering of media attention and their influence with many policy makers, reports released by private think tanks vary tremendously in their quality. Many think tank reports are little more than ideological argumentation dressed up as research. Many others include flaws that would likely have been identified and addressed through the peer review process. We believe that the media, policy makers, and the public will greatly benefit from having qualified social scientists provide reviews of these documents in a timely fashion." He adds, "we don't consider our reviews to be the final word, nor is our goal to stop think tanks' contributions to a public dialogue. That dialogue is, in fact, what we value the most. The best ideas come about through rigorous critique and debate."
**********
###
The Education and the Public Interest Center (EPIC) at the University of Colorado at Boulder and the Education Policy Research Unit (EPRU) at Arizona State University collaborate to produce policy briefs and think tank reviews. Our goal is to promote well-informed democratic deliberation about education policy by providing academic as well as non-academic audiences with useful information and high quality analyses.
Visit EPIC and EPRU at http://www.educationanalysis.org/
EPIC and EPRU are members of the Education Policy Alliance
(http://educationpolicyalliance.org).
###
**********