A new report from the U.S. Government Accountability Office (GAO) provides one of the first large-scale comparisons of special education enrollment between charter and regular public schools. The report’s primary finding, which, predictably, received a fair amount of attention, is that roughly 11 percent of students enrolled in regular public schools were on special education plans in 2009-10, compared with just 8 percent of charter school students.

The GAO report’s authors are very careful to note that their findings merely describe what you might call the “service gap” – i.e., the proportion of special education students served by charters versus regular public schools – but that they do not indicate the reasons for this disparity.

This is an important point, but I would take the warning a step further:  The national- and state-level gapsthemselves should be interpreted with the most extreme caution.

Although there are plenty of interesting data contained in the report, and its authors do point out many of the limitations of their analyses, the GAO’s approach (and virtually all the press coverage) seems to gloss over the fact that charter schools are disproportionately located in urban, lower-income areas, where special education rates of all schools tend to be higher.

To understand why this matters, consider an analogous example: On average, U.S. charter schools serve a considerably larger proportion of minority students than regular public schools. Does this mean that minority students are underrepresented in regular public schools?

No, not really. The primary reason why charters serve a larger proportion of minorities is because they happen to be concentrated in areas, such as big cities, with larger minority populations. Simply comparing the overall raw percentages doesn’t tell us much, since it’s largely a function of where schools are located.

The same goes for special education. If you want to estimate the “service gap” between the two types of schools, you have to at least try to account for the pool of students that both types of schools serve – e.g., by controlling for student characteristics, comparing charters and regular public schools within the same area, or, ideally, comparing charters with their “feeder” schools. This makes national or even state-level analyses difficult, due to data availability, but it really is necessary if you want to get a sense of the degree to which special education students are over- or underrepresented in charter schools.

(In fairness, the report does include some non-specific descriptions of findings from comparisons within metropolitan areas, which are still limited but far more useful, and the report also mentions that GAO did perform limited district-level comparisons, but no results seem to have been presented. The GAO will hopefully expand upon these analyses in future reports.)

Let’s quickly illustrate these issues using data from a single state – Ohio.

If you calculate the 2010-11 statewide weighted average of special education students in Ohio charters versus regular public schools, it’s roughly equal – around 14.7 percent in regular public schools, compared with 14.4 percent in charters (the GAO report has the difference at two percentage points, perhaps due to differences in our data and the year to which they apply).

However, three-quarters of Ohio charter students are enrolled in schools “hosted” by one of the state’s “Ohio 8” districts – Akron, Canton, Cincinnati, Cleveland, Columbus, Dayton, Toledo and Youngstown.

Students in these districts are more likely than their counterparts in the rest of state to be low-income, non-native English speakers and, of course, in special education programs. A statewide comparison simply ignores this, and thus provides a potentially misleading picture of the size of the special education “service gap.”

One rough way to get a better (albeit still highly imperfect) sense of the Ohio gap is to simply compare the rateswithin districts, as in the graph below.

(Note: Negative numbers mean charters serve smaller proportions of special education students. Also, there are only five charter schools in Canton, so I did not disaggregate it.)