Charter schools, though they comprise a remarkably diverse sector, are quite often subject to broad generalizations. Opponents, for example, promote the characterization of charters as test prep factories, though this is a sweeping claim without empirical support. Another common stereotype is that charter schools exclude students with special needs. It is often (but not always) true that charters serve disproportionately fewer students with disabilities, but the reasons for this are complicated and vary a great deal, and there is certainly no evidence for asserting a widespread campaign of exclusion.

Of course, these types of characterizations, which are also leveled frequently at regular public schools, don’t always take the form of criticism. For instance, it is an article of faith among many charter supporters that these schools, thanks to the fact that relatively few are unionized, are better able to aggressively identify and fire low-performing teachers (and, perhaps, retain high performers). Unlike many of the generalizations from both “sides,” this one is a bit more amenable to empirical testing.

A recent paper by Joshua Cowen and Marcus Winters, published in the journal Education Finance and Policy, is among the first to take a look, and some of the results might be surprising.

Here is a super-quick summary of the approach: The authors utilize a pretty extensive dataset, which includes students and teachers from Florida elementary schools, both charter and regular public, spanning 2002 to 2008. They calculate teachers’ value-added scores, and then examine the relationships between these performance measures and teacher exits (as well as intra- and inter-district mobility), controlling for a range of factors, including observable teacher characteristics such as salary, experience, race and gender, as well as school-level characteristics such as student poverty and average achievement, etc. Note that the exits include both voluntary and involuntary leavers – that is, teachers “leave” the Florida dataset, but it’s not possible to determine why.

There are three main findings.

First, charter school teachers are more likely to exit than similar teachers working in comparable regular public schools. This squares with the prior research (also see here and here, or our discussion), and suggests that the higher attrition rates in charters aren’t explained away by differences in teacher/school characteristics.*

Second, when performance is gauged with value-added estimates, lower-performing teachers are more likely to exit than higher-performing teachers. This is also generally consistent with prior research looking at the relationship between attrition and student test-based productivity measures in regular public schools (see here, here and here), thus suggesting that the same situation may be found in charters.

(Side note: These relationships do not generally hold up when it comes to predicting teachers who change schools within or between districts, though there is some indication that teachers with higher value-added in math may be more likely to transfer. In general, though, these types of moves were relatively rare among the teachers in the dataset.)

But the third main finding is the most interesting, and it’s the interaction of the first two: There is no significant charter/regular public school difference in terms of the relationship between (test-based) effectiveness and the likelihood of exit. In other words, performance does not predict exits from the dataset any better in charters than in district schools.

This does not support the common argument that collective bargaining agreements impede the exits of ineffective (i.e., higher value-added) teachers in regular public schools. If that was the case, one would expect to find a stronger relationship between value-added and the likelihood of exit in schools that do not have those contracts. The results suggest otherwise, leading Cowen and Winters to conclude: “Whatever administrative or organizational differences may exist in charter schools, they do not necessarily translate into a discernible difference in the ability to dismiss poorly performing teachers.”

It’s important to avoid over-interpreting the results of this paper. Most basically, there’s a difference between not finding evidence for something and disproving it. In addition, of course, this is just one set of charters being compared with one set of regular public schools in a single state, using one particular (and contentious) measure of effectiveness (as well as a somewhat limited attrition proxy). The relationships between teacher attrition, employment conditions and performance (however measured) are complicated and vary by context; there are a lot of moving parts here. It’s also worth noting that the available evidence suggests there is no difference in performance between Florida’s charter and regular public schools.

We’ll see whether the findings discussed above are corroborated by future work using data from other charter markets (particularly those in which charters perform well). In the meantime, they might serve as yet another reminder that much of the received wisdom – on all “sides” of the education debate – consists of still-unanswered empirical questions, and that it’s usually wise to keep an open mind no matter how certain you might feel.

- Matt Di Carlo

*****

* However, when the sample is limited to teachers with fewer than four years of experience, the likelihood of exit, all else being equal, is not drastically different between the sectors. The authors speculate that the tenure process may have a “winnowing effect” on the least effective probationary teachers in regular public schools, while the same process occurs “organically” in charters.