Dear Deborah,

Last June, I wrote an op-ed for The New York Times disputing the idea of "miracle schools." With the assistance of two volunteer researchers, Gary Rubinstein and Noel Hammatt, I learned that several schools touted by various political leaders as miraculous were not. My intention was not to criticize the schools and their staff, but to criticize the politicians who were using the schools to imply that their policies (like firing the staff and closing the school) were working and that it wasn't all that difficult to turn around a school that enrolled large numbers of low-performing students.

The politicians seemed to suggest that their policies (testing and accountability or mass firings) sufficed to produce dramatically higher test scores and graduation rates. The subtext is that poverty and resources are not actually problems for urban schools; if they could just test more often and fire more teachers, the corporate reformers imply, then test scores would soar. This analysis suggests that schools enrolling the neediest students do not need more resources, and it rationalizes the current trend of draconian budget cuts for public education—for the arts, pre-kindergarten, libraries, physical education, and other non-tested subjects and services.

Soon after my article appeared in the Times, Newsweek published a story hailing 10 "miracle" schools. This seemed to be a direct response to my article. Gary Rubinstein and Noel Hammatt investigated the Newsweek 10 and disqualified them as "miracle schools" because they did not meet one or more of the following criteria:

1) A low attrition rate
2) High test scores
3) High graduation rate (for high schools)
4) High college acceptance rate (for high schools)
5) Fair representation of English-language learner (ELL) and special education students
6) A high percent of students who qualify for free or reduced meal prices
7) Funding equivalent to the nearby 'failing' school
8) No evidence that the school discriminates against low-performing students

Gary, a blogger, Teach for America alumnus (and critic), and high school mathematics teacher, became so interested in the miracle school phenomenon that he created a website to publish reviews of miracle claims.

Last week, Gary debunked a story that was prominently featured in Education Week about a "turnaround" school, the Academy@Shawnee in Louisville, Kentucky. This story seemed to validate the punitive policies advanced by U.S. Secretary of Education Arne Duncan. But when Gary examined the school's record, he found a very different story from the one reported in Education Week.

Then Gary reviewed the miracle claims of the New Orleans Recovery School District. New Orleans has been widely touted as proof that incredible results can be achieved by getting rid of the teachers' union and converting most schools into privately managed charter schools.

Two days ago, the oft-told tale was repeated in an editorial in The New York Times. The editorial begins with the statement that, before Hurricane Katrina, more than 60 percent of New Orleans' students attended a failing school, and now only 18 percent do. Among the unasked questions: Are the students in New Orleans the same ones who were in the schools before the hurricane? How many of the city's poorest children returned? What is the definition of a "failing school"? Was the definition the same pre- and post-hurricane? What methods are the presumably better schools using to produce such miracles?

Once again, Gary found that the hype exceeded reality. By examining state data, Gary learned that the district is in fact one of the lowest-performing in the state of Louisiana. In fact, it is 69th of 70 districts. The state gave a D or an F grade to 87 percent of the schools in the Recovery School District. Its much-heralded "improvement" is based on a statistic that exaggerates growth for districts with low baseline scores.

The lesson in all this debunking is not that poor kids can't learn. Of course, they can. Let me say that again, slowly: Yes, poor kids can learn and excel. But whether or not children are poor, education is a slow, incremental process. While it is true that a student may have a remarkable change in attitude and motivation and demonstrate large test-score gains in a short period of time, it is rare indeed when an entire school or district experiences a dramatic increase in test scores. Any huge change in scores for a school or a district in a short period of time ought to provoke skepticism and a demand for evidence, not a willing suspension of disbelief.

Like you, I don't believe that test scores are by themselves a genuine proxy for achievement because test scores may indicate nothing more than a heavy investment in test prep. As Daniel Koretz points out in his valuable book Measuring Up, too much test prep may compromise the value of the measure. I used to think that test scores were a reliable gauge of academic achievement. Now I take care not to confuse the two. Not only have we seen widespread evidence of cheating and gaming the system, but it seems obvious that the over-use and misuse of standardized testing is distorting the educational process, narrowing the curriculum, and conflicting with the goals of meaningful education.

But as long as public officials insist on making test scores the measure of teacher quality and school success, then their claims should be closely scrutinized using the metrics that they themselves have made the coin of the realm. Many of the schools that politicians hail as successes have records no different from other schools that the politicians are closing.

I worry that our current national obsession with test scores has spiraled out of control and is harming students, teachers, principals, and the quality of education. How will we regain our common sense?

Diane