A couple of weeks ago, the website Vox.com published an article entitled, “11 facts about U.S. teachers and schools that put the education reform debate in context.” The article, in the wake of the Vergara decision, is supposed to provide readers with the “basic facts” about the current education reform environment, with a particular emphasis on teachers. Most of the 11 facts are based on descriptive statistics.

Vox advertises itself as a source of accessible, essential, summary information — what you “need to know” — for people interested in a topic but not necessarily well-versed in it. Right off the bat, let me say that this is an extraordinarily difficult task, and in constructing lists such as this one, there’s no way to please everyone (I’ve read a couple of Vox’s education articles and they were okay).

That said, someone sent me this particular list, and it’s pretty good overall, especially since it does not reflect overt advocacy for given policy positions, as so many of these types of lists do. But I was compelled to comment on it. I want to say that I did this to make some lofty point about the strengths and weaknesses of data and statistics packaged for consumption by the general public. It would, however, be more accurate to say that I started doing it and just couldn’t stop. In any case, here’s a little supplemental discussion of each of the 11 items:

1. Teachers earn a bit more than the average American. This fact presents a simple comparison, using BLS data, of median elementary and secondary school (public and private) teacher earnings with those of the median U.S. household. There are a few issues here (most are mentioned in the article). First, I have no idea why, but this is comparing an occupation-level figure (for teachers) with a household-level figure. Second, it is comparing teachers’ earnings with those of all workers (households), rather than workers with similar education and experience. Third, the comparison omits non-salary compensation (i.e., benefits), which are an important part of this picture. These caveats, the first two in particular, make for a less-than-meaningful contrast. Overall, this illustrates how seemingly basic comparisons, such as earnings between occupations, can be painfully complicated - even using more sophisticated methods, it depends on how compensation is measured and the choice of comparison groups. It also bears mentioning that, for some advocates, the big issue in education today is as much about how teachers are paid as about how much they are paid.
2. Americans say teaching is a prestigious career. This item uses polling data to compare how people perceive the prestige of the teaching profession (quite highly) with that of a selection of other occupations. I think this is a good one, especially given all the heated rhetoric (from both sides, ironically) about “reprofessionalizing” or “deprofessionalizing” teaching, or about how young people avoid teaching due to its reputation. In reality, most people hold the profession in high regard. Also, as a side note, if you go to the original report from Harris Interactive (the data source link in the article seems to be incorrect), it’s very interesting to see that the proportion of respondents saying teaching has “very great prestige” almost doubled between 1977 and 2009 (from 29 to 51 percent).
3. Elite students tend to avoid teaching. Using data from a widely-cited McKinsey report, this one points out that incoming teachers’ SAT/ACT scores are lower than those of graduates who pursue other careers (in terms of the common “top/bottom third” figures, which we discuss here), and that this is not the case in a few other nations that receive attention for their high test scores. On the substance, as is discussed in the post linked above, SAT/ACT scores are among the only measurable pre-service characteristics that has a record of being associated with value-added scores once graduates start teaching (which is, of course, just one measure of effectiveness), but the association is not particularly strong. Overall, predicting who will be a good teacher before they enter the classroom remains remarkably elusive, and I think that generalization, while not a “fact” in the same sense as are descriptive statistics, is probably the most important thing to keep in mind when talking about attracting the “best candidates” to the profession.
4. Absolute teacher salaries are high. Here, based on OECD data, we see that teacher salaries throughout their careers (the graph shows salaries after 10 years on the job) are a bit lower than they are in other OECD nations. As was the case with the U.S. figures, this comparison fails to account for differences in the cost of health and pension benefits, but that’s just a quibble (and tough to compare between nations). Note, though, that the evidence that salaries have an impact on the quality of candidates or whether they remain is not entirely consistent – it varies by context and other factors (see the papers cited in this post).
5. Relative teacher salaries are low in the U.S. This compares teacher salaries to OECD nations as a proportion of college graduates’ salaries. I like the idea of at least trying to compare teachers with similarly-educated workers (which was not done in #1), although I would reiterate that the role of salary in who pursues and remains in teaching, at least in the U.S., is not as clear as one might think. 
6. Some teachers are much more effective than others. The flagship piece of evidence here is a graph from a published article by economist Eric Hanushek, one that is based on a rough, illustrative calculation that expresses teacher effects in terms of classroom-level lifetime earnings (thus making for huge numbers as one moves toward the right of the [unlabeled] class size horizontal axis). In my view, there are better ways to express this important finding about the variation in teacher effects that also would have been accessible to people not accustomed to thinking in terms of standard deviations. That said, this is probably the most important point among the 11 presented in this article, if for no other reason than the sheer magnitude of its impact on the debate and policymaking.
7. Class sizes have fallen substantially. This item presents declining student/teachers ratios alongside the number of teachers in the U.S. since 1960, using NCES data. These ratios are a crude measure of class size (more teachers per student doesn’t translate directly into smaller classes, particularly given the sharp increase in special education teachers that occurred within this time period). The size and scope of the trend, however, are sufficient to make the point that class sizes have decreased over the long-term, even if the decline may be more moderate than is suggested by the graph. This is an important point in the context of contemporary education reform in the U.S. (but it might have been helpful to discuss briefly some of the research on this topic).
8. The teaching workforce is disproportionately unionized. Here, BLS data are used to show that unionization rates for the occupational group “Education, library and training workers” are higher than the rates for local government, public sector workers , and private sector workers. As the article notes, the BLS website does not offer teacher-specific unionization rates, but there are reputable resources that produce such estimates (using BLS data). They show that between 45-55 percent of teachers are union members and/or covered by a contract. Note, however, that these figures, like the category-wide statistic presented in the article (Education, library and training workers), include both public and private schools teachers (and thus understate unionization rates, since the latter group is largely non-union). Still, all of these sources make roughly the same point– teachers are more heavily unionized than most other U.S. occupations, particularly those in the private sector. Of course, the big issue in education is not really whether teachers are unionized, but rather whether unions, and collective bargaining in particular, have a large impact on school performance. This issue is (predictably) complicated and largely unresolved. 
9. Per pupil funding varies widely. A simple color-coded U.S. map is used to illustrate funding disparities between states. As was the case with the class size, I think this (i.e., funding) is an important general topic, even if raw per pupil figures, though easy to understand, aren’t necessarily the best measure in this context, given that student characteristics vary quite widely between states, and they play a major role in determining funding at the federal, state and local levels. Perhaps including a funding fairness or similar measure, or at least mentioning them, would have been better. Nevertheless, funding does matter, and the article’s point — that it varies between states — is certainly important.
10. Dropout rates are falling. The chart in the article shows dropout rates falling since 1970 for different student subgroups. Again, this is a tough outcome to measure nationally. More importantly, it seems redundant with #11.
11. Students are broadly doing better. This is a simple graph of trends in Long-Term NAEP scale scores by age and subject (math/reading) since 1971. The scores are clearly increasing. This is an important point to make in any list of this type, and these are good data with which to make it. The only thing I would add — and it’s no less important — is that the degree to which these trends are due to improvements in schooling, to say nothing of teacher quality, rather than the dozens of other factors that affect testing performance (including change in the sample of test takers), is unclear. No question, though – this one is “need to know.”

Overall, then, this list is generally well done, even if reasonable can disagree on what should and should not be included, as well as on the measures chosen to make the points.

Finally, rather than just pick apart somebody else’s list, it’s only fair that I put forth a few suggestions that I think should have been on a list of what you “need to know” about the education reform debate, with an emphasis on teachers. There are so many possibilities, but here are a few off the top of my head.*

1. Teacher experience is among the only measurable characteristics associated with (test-based) effectiveness, but improvement tends to level out after a few years. Given all the policy and controversy surrounding teacher seniority in multiple contexts (pay, layoffs, etc.), this generalization of the findings from research on experience (see the papers cited in this post) would seem to be something of which people should be aware (though, as usual, that research is almost entirely focused on test-based effectiveness as the outcome). The flip side of this generalized finding is that the vast majority of other widely-available measures, including master’s degrees and certification, tend not to be associated with test-based effectiveness, which is the major basis for the push to change compensation and other systems.
2. Teachers generally support some of the more controversial current reform efforts in broad strokes, but can be skeptical of the details. Data from this claim can come from a combination of several national teacher surveys (e.g., those from Education Sector, Public Agenda, Scholastic, etc.). The big point here is that teachers are not necessarily unreceptive to reform or accountability, but that they sometimes oppose the current direction of policy in terms of the details. On some issues, such as class size and funding, teachers are relatively consistent in their support. On many other issues, however, particularly the more controversial issues being discussed heavily over the past 10-15 years, things are more complicated. For example, as shown here, teachers are generally supportive of trying out alternative compensation systems, but they are overwhelmingly opposed to using test scores as the basis for the new systems. A simple breakdown of a few of these questions would demonstrate the importance of the proverbial “fine print” in designing and implementing new policy, as opposed to the lofty rhetoric (from both “sides”) that tends to dominate the debate.
3. Nationally, teacher attrition increased between the early 1990s and mid-2000s, but it may be declining now (and it varies greatly within and between locations). The data for this claim are summarized in this post. This simple trend is important to the education reform debate, given all the attention that policies such as test-based accountability receive in the context of how they might affect teachers’ labor market behavior. And I think it also bears mentioning here that such behavior is probably as much affected by labor market conditions as by educational policy per se.
4. Different measures of teacher performance are significantly but not strongly associated with each other. This is one of the most important lessons being learned from the new generation of teacher evaluations – i.e., that common teacher performance measures, most notably growth model estimates and classroom observations, are correlated with each other, but not strongly so (see this excellent summary of the research). To some degree, this reflects imprecision in each indicator – even perfect measures of teacher performance might not “match up” strongly given their imprecise measurement (e.g., due to small samples). But it also suggests that different measures capture different things, which is important not only for policymaking, but also to bear in mind when one hears ostensibly evidence-based claims about “teacher performance” or “teacher effectiveness,” usually referring exclusively to test-based measures (e.g., see #6 in the list above).
5. Only a very small percentage of students attend charter schools, but their market share is growing, and they are disproportionately located in large urban districts. Granted, this one is not directly relevant to teachers, but the debate over charter schools tends to focus on their human resources policies, so many discussions of charters in a sense boil down to teacher quality. Organizations such as the National Alliance for Public Charter Schools gather these data annually. The point here is that charter schools, which receive a great deal of attention and are the subject of intense controversy in education debates, are a relatively minor player on the national scene, but their presence is growing, and is much stronger in the types of districts (e.g., large urban areas) that are focus of many improvement efforts. So, for example, they might have strong influence when it comes to trying out policies, but not so much in terms of their ability to affect national trends such as testing performance (at least not yet).
6. Teacher evaluations that include, at least to some extent, test-based productivity measures are now the rule in the U.S., rather than the exception (but most teachers don’t “qualify” for them). Thanks in large part to Race to the Top, most states are in the process of implementing or have already implemented new evaluations that require districts to base part of teachers’ final ratings on growth model estimates. In other words, these evaluations are the new status quo in U.S. public education. On the other hand, it is crucial to bear in mind that value-added and other growth model estimates are only possible, at least for now, for a minority of teachers – i.e., those that teach in tested grades and subjects. Thus, even in states and districts with new systems that require some role for test-based measures, most teachers’ evaluation scores are based predominantly on non-test measures, most notably classroom observations.
7. Teacher effectiveness seems to vary quite widely, but there is still relatively little evidence regarding how to improve it. For this one, I’m breaking my own rule of sticking with “hard” facts such as descriptive statistics. The point here, as discussed in this post, is that there is an impressive and growing body of evidence suggesting that there is a big difference between “top” and “bottom” teachers in terms of their measured impact on student outcomes (mostly test-based outcomes), but this fact, by itself, is only a motivation for reform, not a concrete policy argument. What matters is how the distribution of teacher quality, regardless of how it is measured, can be improved via interventions (e.g., new evaluations, compensation systems, etc.). And, on this score, there is still precious little evidence. This point is so central to understanding the current state of teacher-focused education reform, and many of the disagreements therein.

- Matt Di Carlo

*****

* Please note that I am sticking with the approach of the article – that is, I am limiting my suggestions to quantitative “facts,” rather than general statements such as “predicting teacher effectiveness using pre-service characteristics is very difficult” (even though I think those types of warnings are in some cases more useful to know).