One claim that gets tossed around a lot in education circles is that “the most effective teachers produce a year and a half of learning per year, while the least effective produce a half of a year of learning.”

This talking point is used all the time in advocacy materials and news articles. Its implications are pretty clear: Effective teachers can make all the difference, while ineffective teachers can do permanent damage.

As with most prepackaged talking points circulated in education debates, the “year and a half of learning” argument, when used without qualification, is both somewhat valid and somewhat misleading. So, seeing as it comes up so often, let’s very quickly identify its origins and what it means.

This particular finding is traceable to a 1992 paper by economist Eric Hanushek, one which focused primarily on the relationship between achievement and family composition in Gary, Indiana (the data are from the early- to mid-1970s, and include only low-income students). After reviewing his (very interesting) main results on the relationship between student achievement and family size, birth order and the interval between births, Hanushek presents an analysis of test-based teacher effects.

Put simply, his results show that adding a teacher effect variable to his models increases their explanatory power substantially, and that the difference between the gains of students with a highly-scoring teacher (in this case, at the 84th percentile) and a lower-scoring teacher (16th percentile) is roughly equivalent to one full grade level (i.e., the difference between half a year and a year and a half). Consistent with the prior and subsequent literature, he also finds that teacher effectiveness is not well explained by “traditional” measures such as teacher education (Hanushek also quickly explores the relationship between family characteristics and teacher effectiveness, which would presumably arise due to search behavior [as it turns out, there is little evidence for this, at least in these data]).

The reflexive reaction to this sourcing might be to dismiss the claim entirely, as it is based on a single analysis of a particular set of students in one place 40 years ago. In a limited sense, that’s fair enough – it’s obviously true that low-income students and their teachers in Gary, Indiana during the 1970s cannot be generalized to the rest of nation in 2012.

But it is, as always, much more complicated than that, as there are dozens of studies finding wide variation between the “top” and “bottom” teachers in any given year, at least in terms of test-based productivity (see here).

The size of these estimated discrepancies depends on so many factors, including (but not limited to): the subject (in general, there is larger variation in math than reading); the models and tests used; the years of data available; and the choice of comparison groups (definition of “top” and “bottom”). Sometimes the differences are equivalent to 4-5 “months of learning,” sometimes they are not (especially in reading).*

So, here’s the deal (and this is strictly my opinion): There is a research consensus that estimated test-based teacher effects vary widely between the top and bottom of the distribution, but the “year and a half” assertion should probably be put out to pasture, at least when it’s used without elaboration or qualification.

It implies a precision that belies the diversity of findings within the research literature, and it ignores the importance of context, data availability, variation between test subjects, etc. There are plenty of ways to express the fact that teachers matter without boiling a large, nuanced body of evidence down to a single effect estimate.

Accessible generalizations certainly have their role in policy discussions, but oversimplification has really crippled the debate about value-added and other growth models, on both “sides” of the issue.

One final note: The truly important point about the “year and a half of learning” argument, just like the ubiquitous “three great teachers in a row can close the achievement gap” talking point, is what they mean for policy (especially since they’re usually used to make policy arguments). They are both stylized ways of saying the same thing, about which there is really very little disagreement – teachers are important. But, even if you take these points at face value, they do little to help answer the critical question that comes next, which is how the distribution of teacher quality can be improved. This is inarguably one of the most urgent issues facing education policy today, from which arguments about these talking points may just serve as a distraction.

- Matt Di Carlo