The situation with vouchers in Louisiana is obviously quite complicated, and there are strong opinions on both sides of the issue, but I’d like to comment quickly on the new “accountability” provision. It’s a great example of how, too often, people focus on the concept of accountability and ignore how it is actually implemented in policy.

Quick and dirty background: Louisiana will be allowing students to receive vouchers (tuition to attend private schools) if their public schools are sufficiently low-performing, according to their “school performance score” (SPS). As discussed here, the SPS is based primarily on how highly students score, rather than whether they’re making progress, and thus tells you relatively little about the actual effectiveness of schools per se. For instance, the vouchers will be awarded mostly to schools serving larger proportions of disadvantaged students, even if many of those schools are compelling large gains (though such progress cannot be assessed adequately using year-to-year changes in the SPS, which, due in part to its reliance on cross-sectional proficiency rates, are extremely volatile).

Now, here’s where things get really messy: In an attempt to demonstrate that they are holding the voucher-accepting private schools accountable, Louisiana officials have decided that they will make these private schools ineligible for the program if their performance is too low (after at least two years of participation in the program). That might be a good idea if the state measured school performance in a defensible manner. It doesn’t.

The private schools’ performance will be assessed by a measure (the “Scholarship Cohort Index”) that is essentially the same as the SPS – it measures how highly students score on tests, not whether they improve. The formula will only be applied to the voucher students, not to the private school students who pay normal tuition (who don’t take state tests).

In case you’re missing the irony: Public schools are opened up to vouchers based on outcomes (absolute performance levels) they largely cannot control, and their students are sent to private schools that are also held accountable for outcomes that are mostly out of their hands.

In other words, it’s to no small extent a crapshoot. For example, if a private school accepts 50 voucher-bearing students over a two year period, and those students happen to be sufficiently high-scoring to make the cut, regardless of whether or not they improve, the private school will remain eligible for the program. If not, then they may very well be deemed unfit to accept vouchers because they got “unlucky” in the students who used them.*

Further complicating the situation, there’s a loophole of sorts: The performance requirement only applies if the private schools enroll more than 40 tested voucher students, or more than 10 students per grade. This exemption has caused a great deal of outrage among critics who think that private schools should be held to the same standard as their public counterparts. Supporters, in contrast, say that the more vouchers a given school accepts, the more accountable it should be.

Again, in this particular instance, both arguments may be missing the point, as they both assume that the performance measure is valid for the purpose of assessing school effectiveness. It’s not. Private schools shouldn’t be held accountable for outcomes they can’t control any more than public schools should, regardless of how many vouchers they accept.

No matter what you think of Louisiana’s voucher program, the state’s measure (SPS) largely assesses student, notschool performance, and thus the program’s accountability rules are unlikely to work as advertised. The allocation of vouchers, whether to students or schools, will have very little to do with actual school quality.

- Matt Di Carlo

*****

* Even if the state employed a more rigorous measure of actual school performance, it would be difficult to assess their actual effectiveness using such small samples. This is not Louisiana’s “fault;” the dispersion of voucher-bearing students among private schools would be an issue in almost any voucher accountability regime.