One of the more annoying things about discussing and debating education policy is seeing how misconceptions are born. You can often trace an incorrect notion from a think tank's brief to a reformy "analyst" who only read the press release and wouldn't understand the methods section (if there is one) anyway. That supposed "expert" than passes it to the larger punditocracy, who think it's a good idea to write about many things shallowly, rather than regularly inviting actual experts to opine in their stead.

People like me then bang our heads against our keyboards when we read something in the media that's just not true. If we're obsessive (*ahem*), we trace the idea back to its origin (which often requires detective work, as pundits love to spew "facts" without actually saying where they came from), where it's either found to be the product of misinterpretation, or just plain wrong at the source.

Case in point:

We got a few stories last month about how charter schools don't really get all that much philanthropic revenue. "Analysts" tell their readers that people like me are just blowing smoke when we point out that philanthropic giving is a great help to certain charter operators and should be taken into account when ill-informed pundits decide to write stories about those charters' jewel-like shine.

The genesis of this notion that charitable giving to charters doesn't matter is a report by the University of Arkansas's Department of Education Reform (yes,there really is such a thing, I swear). "Buckets of Water Into The Ocean: Non-Public Revenue in Public Charter and Traditional Public Schools" is the latest in a series of briefs from U-Ark that purport to show that, compared to public district schools, charters are barely scratching by on a fraction of the revenue their host districts rake in.

When U-Ark released an updated report on this topic in 2014, Bruce Baker* took it apart in a definitive brief for the National Education Policy Center. Baker's work speaks for itself, and I won't try to encapsulate it all. But there are, according to his brief, at least three big problems with U-Ark's methods:

1) U-Ark compares per pupil revenue in charters with per pupil revenue in districts -- statewide.This is just plain old dumb. Charters are largely clustered in urban areas, and schools there should not be compared to schools across an entire state, including suburban and rural schools. The student populations are not the same, and the revenues are not the same. It's just a totally meaningless comparison.

2) The report is "alarmingly vague" in describing its methods and data sources.But that's pretty much the standard these days for reformy research.

3) U-Ark only looks at revenue without taking into account where it is spent and on whose behalf. This is a huge, huge flaw. Let me ask Bruce's indulgence here a bit and try to state the problem in layman's terms:

For this example, we'll talk about transportation. In many states (including New Jersey), school districts that host charter students must provide transportation for those students attending those charters. This saves the charter from having to worry about getting their students to school, both financially and logistically, as the entire burden is on the local district.

Here we see the taxpayers giving money to the local district for transporting all of the district's resident students, whether they attend a charter or a district school. The charter school gets none of it... but then again, they don't have to transport any kids! But U-Ark doesn't appear to care at all about this:

Instead, because they are focusing only on revenues, they look at the extra money the districts get without thinking about the extra obligations the district has. Obviously, this makes no sense; there has to be some way to account for the difference if we're going to make a useful comparison. Unfortunately, as Bruce points out, there is no evidence U-Ark ever attempted to do just this.

So here's the funny thing about this latest report: rather than show that they did make adjustments and just didn't document them, or adjust their methods to correct their previous mistake, U-Ark instead revels in their lack of appropriate methodology:

Other critics of our report counter that our methodology is flawed because we count as district revenue the funds received by districts that pass through the district to area charters. Traditional public schools often receive funds that are, in turn, given to charter schools. Failing to account for this funding mechanism would overstate the amount of revenue TPS receive. Critics also argue that charters might enroll a higher proportion of reduced-price lunch students than TPS but that TPS enroll a higher proportion of the very poor students who qualify for free lunch (Baker 2014). This charge that we count pass-through charter revenue as district revenue is false. We count all revenue based on where it ultimately ended up, as documented in audited originally (see our methodology section below). The claim that charters enroll a lower proportion of free-lunch eligible students than TPS is also incorrect based on NCES data (Wolf et al., 2014).

Some researchers have additionally criticized our school revenue study for not focusing on school expenditures (Baker 2014). We maintain that a revenue study should focus on revenue, as the total revenue that an educational organization receives represents the actual amount of resources that are committed to that organization, regardless of how those resources are subsequently spent.* If one were interested in the total amount of federal taxes paid by Americans in a given year, one should not look at the total amount of federal government expenditures in that year because revenues are not the same as expenditures. [emphasis mine]

That is a preposterous analogy. U-Ark isn't looking at total revenues in all publicly-funded schools; it's dividing up the schools into groups that do different jobs. What they're actually doing is more like counting up the revenues that flow to the Defense Department and the Department of the Interior, than bemoaning the fact that one gets less -- without acknowledging that they don't have the same function!

Here's another example:

This time, the taxpayers are funding free and reduced-price lunches (FRPL). But let's suppose the charter school is contracting out its lunch service to its host district (from what I can tell, this seems to be a fairly standard practice, particularly in co-located charters). It passes through its federal lunch subsidy to the district, which, in turn, provides the charter students with their meals.

But did U-Ark count that pass-through as revenue for the district? It sure seems like they did (again, the methods are so poorly documented it's hard to be sure), and that would be a mistake. The district is providing a service to the charter students for those funds, not the district students. Yes, it's extra revenue, but it's also extra responsibility.

So this is the fundamental problem with U-Ark's research, even before we get into the topic of non-public revenue -- which, contrary to the apparent limited understanding of our reformy "analysts," is not defined by U-Ark as solely as philanthropic giving. In fact, the largest source of non-public funds for district schools, aside from "miscellaneous" funds, is food service.

Here we see the flow of non-public funds into and out of schools for paid lunches -- including the suburban schools (we'll see why this is important in a minute). If the charter contracts out with the district for its food service, the charter students who pay for lunch will actually be paying the district. As regular readers here know, charter school students are less likely to qualify for free lunch than students in their host district schools, which means the urban school is getting proportionally more of its non-public lunch revenue from charter students than from its own students.

Maybe the charter student pays the charter, and it passes through the funds. Does U-Ark double count this? They say they don't: "We count all revenue based on where it ultimately ended up." OK... but then, once again: the district school does get more revenue, but it also has more responsibilities. It has to make extra grilled cheese and tomato soup for the charter kids -- that's why it got the money!

In addition: out in the suburbs, very few kids are getting free lunches. Those districts are also making lunches, but the revenue to make those meals is coming from non-public sources, not the feds. And here's where U-Ark's bad habit of aggregating all districts in a state screws up their analysis.

All of a sudden, all the urban and suburban districts are lumped together and compared to the charters. As I've said many times, the charters don't have as many free lunch kids as their urban hosts -- but they still have way more than suburban schools. So comparing urban charters with suburban district schools makes it look as if all district schools are swimming in non-public lunch money!

This is just nuts. The suburban schools are providing a service for all this non-public revenue, yet U-Ark pretends as if they are getting extra revenue, even though that brings with it extra responsibilities. Worse, the urban schools get dinged, even though they only have a small number of students who pay for their lunches!

But hang on -- it gets even more ridiculous. Because the U-Ark folks have decided to treat lunch revenue and tuition revenue and rental revenue and all sorts of other sources of funds as equivalent to philanthropic giving; they're all just different sources of non-public revenue.

But they aren't equivalent. When a school receives a charitable donation, there is no obligation to provide a direct service in exchange for that money. If a kid buys a lunch from a school, the school gives that kid a lunch. If a theater company rents the school auditorium over the weekend, the school keeps the lights on and pays its custodian to clean up the mess. 

A philanthropic donation is not the same sort of transaction. Yes, the donation may be earmarked for a particular item, but that item is helping the school in its primary mission; it's not an "extra" outside of what the school is set up to do. So the aggregation of philanthropy with things like lunch revenues is not in any way useful.

But even when U-Ark does separate out philanthropy, the way they do it makes no sense.

Suburban charitable giving for schools is rarely in the form of direct cash grants to districts. Local residents will instead set up "education foundations," which can be a big source of support for district schools. The foundation has its various fundraising activities -- much of it giving from parents whose children are enrolled in the schools -- and then gives grants directly to teachers for various educational items like smart boards.

As far as I can see, U-Ark entirely ignored this giving. I say this because in Table 5 (p.19), it lists New Jersey district schools as receiving a mere $19,922 in total revenue from philanthropic sources. But the Millburn-Short Hills Education Foundation -- just one suburban district -- gave over $204,000 dollars to its schools just last year. (Didn't any of the study's five authors stop and say, "Uh, guys, this figure just doesn't seem right..."?)

Conceivably, this actually supports U-Ark's contention that the charters are falling behind on non-public revenue. But not so fast: some charters have organizations that assist them in gathering extra revenue, just like the suburban education foundations. You can find the tax statements for many of these "Friends of..." groups on Guidestar. I've been looking at them here in New Jersey (more on this later), and some of the amounts are eye-opening. The Princeton Charter School actually has two organizations supporting it, and the amounts involved are very, very significant.

In addition, some groups like Friends of TEAM (KIPP) Academy Charter School appear to rent facilities to their affiliated school at a loss, a form of non-public subsidy. According to their 2012 Form 990, FOT took a rental loss of $1.8 million (Form 990, Part VIII, line 6c). That's a big chuck of change by any measure. But U-Ark doesn't appear to have included this form of subsidy (again, who can tell?). Instead, their analysis looks like this:

"Buckets in an ocean," huh? Looks like maybe you skipped a few:

Again: conflating suburban and urban districts obscures what's really going on. We could use some more research on the impact of these education foundations; however, it's safe to say that the per pupil charitable giving in an affluent town like Millburn is going to be much higher than the giving in a less wealthy place like Newark. But the charters are in Newark, not Millburn (for now). It pointless to conflate the two, just as it's pointless to conflate lunch revenues with charitable donations.

To be fair: one point that U-Ark does make that bears repeating is that charter philanthropy is not spread evenly across the sector. A few big-name charters appear to get most of the charitable giving; that lines up with previous research, which does not lump together all charter schools. But U-Ark does not make this distinction in their executive summary:

Although charitable funds from philanthropies make up almost half of the non-public revenue in the charter sector, they account for only 2.5% of total charter revenues nationally and therefore cannot be expected to close the 21.7% total funding gap between charters and TPS in these 15 states.

Well, sure -- across the sector. But does philanthropic giving help specific charters pull even when compared to their host districts, accounting for differences in student populations and responsibilities? That is the relevant question -- and the one U-Ark does not ask.

Think tanks (and let's face it, that's what U-Ark's Department of Education Reform is) have got to start doing better. People who push reformy lines have shown themselves time and again incapable of understanding the nuances of issues like school finance and charter school proliferation. They continue to be lead astray by reports like this one from U-Ark, which takes a complex issue and oversimplifies it to the point where the conclusions drawn are simply wrong.

Simply saying: "This is a revenue report!" isn't going to cut it. Raise your game, folks.

Go hogs!

ADDING: There's a truly bizarre footnote in this report that pretty much sums up its major flaw (p. 8):

The one source of funds that we excluded from the total for both charters and TPS is revenue from bond issuances, since those funds have to be repaid. We also excluded revenues for adult and preschool education because our study focused on revenues for K-12 education only. [emphasis mine]

But why would you? You include transportation revenues, even though the districts have to pay for those costs for both charter and district students while the charters are exempt. You include non-public food service revenues, even though, once again, many districts are likely providing a service charters are not.

Why would U-Ark arbitrarily decide that pre-K revenues are not part of a district's total revenue stream? Why exclude this but not transportation? If you take out one, you have to take out the other.

Right?

* As always: Bruce is my advisor at Rutgers GSE in the PhD program in Education Theory, Organization, and Policy.