I was following a conversation on Twitter a short while back in which one student activist – Stephanie Rivera of Rutgers asked another – Alexis Morin from Students for Education Reform – why SFER chooses to focus almost exclusively on charter schools as beacons of “success” and thus a significant part of the “solutions” for urban education moving forward. Observing this interaction brought me back once again to the astounding gaps in logic which are so pervasive in the current reform rhetoric which seeks to find policy solutions almost exclusively in charter schools and in changing teacher compensation and dismissal policies.  The reformy solutions are pretty much a given regardless of the original question or what the analyses yield.

Too often, the following faulty reasoning is applied in search for solutions in the education reform debate:

1. Scan the horizon for successful charter schools (even though charters are no more successful, on average than their more dominant counterparts – district schools). [Charter schools on average, are average. Some are above average and others, well, not so much. Because charters are still much smaller in total numbers, if one was to simply look for "good" schools, the odds of picking a charter would in fact be smaller.]
2. Assume that better-than-average charter schools – successful ones – are better than traditional public schools. [ignoring that while, by virtue of being the upper half of a similar distribution, they are really no different from the upper half of traditional public schools].
3. Assume that because the better-than-average charter schools are better than the average traditional public school, that being a charter school  – bearing the label/classification “charter” – has something to do with it. [even though a comparable - or even larger - share of other schools bearing the label "charter" are actually doing worse than the average traditional public school].
4. Assume that bearing the label – “charter” – necessarily means that these schools have and use creatively and inventively the substantially greater autonomy granted to them. [That is, they certainly don't waste their time on stuff like spending more money, providing smaller class sizes and paying teachers more].

I have seen this utterly ridiculous stream of contorted logic rolled out on numerous occasions in the past few years, and even in the past few weeks.

A while back, I posted two separate entries called “Searching for Superguy” – one for New York City and one for New Jersey – in order to display the distribution of performance, corrected for demographics, for New York City and New Jersey charter schools.

Since that time, I’ve compiled quite a bit more data on charter (and other) schools in a variety of settings. I’ve also developed a clearer vision of exactly what constitutes one of those “miracle” schools we’re all searching for. A miracle school is characterized by at least the following four factors:

1. Serves the same kids (poverty, language proficiency, disability)
2. Spends less than other schools serving similar kids
3. Has high average outcomes compared to schools serving similar kids
4. Achieves better value-added on measured student outcomes than other schools

For today’s post, I offer you a tour of charter schools in New York City and in Houston Texas – two cities with significant concentrations of charter schools and two cities with significant numbers of charter schools affiliated with major charter management organizations.

A Tour of Houston and New York City Charter Schools

Serve the Same Kids?

The first question, of course, is do charter schools in these cities serve the “same kids?” as traditional public schools in the same city/borough and at the same grade level. To answer this question, I estimate regression models to three years of data (2008 to 2010) where the population characteristic of interest is the dependent variable, and a)year of data, b) location of school (city) and c) grade level/range are the independent variables. Houston is treated as a single city (uh… because it is) and New York is carved into boroughs for this analysis. That is, schools are compared with same grade level schools in their borough. Charter Schools are lumped together by CMO, with schools not belonging to major CMOs lumped together for this analysis (they are indeed a very heterogeneous group). The analysis is weighted by the enrollment of the schools.

Figure 1 shows that in New York City, Charter Schools serve a) far fewer children who qualify for free lunch , b) far fewer LEP/ELL children and c) far fewer children with disabilities than school serving the same grade level in their borough. Uncommon schools are indeed the least common. But Success academies have particularly large deficits in LEP/ELL children. These schools simply aren’t comparable in terms of student populations.

Figure 1. New York City Demographics

Figure 2 shows the Houston schools by CMO. With my present data I was unable to parse free from free or reduced price lunch, which may be important. But, this figure and some of my previous analyses confirm that at least in Houston, charters are doing a better job of serving low income kids (than NYC charters). However, most Charter CMOs still seem to have a significant aversion to children with disabilities and in most cases, also to children who are LEP/ELL.

Figure 2. Houston Demographics

Spend Less?

The spending analysis is conducted similarly, using 3 years of data and using a regression model where spending per pupil is the dependent variable and where student characteristics, grade level and year are the independent variables. This analysis plays off our recent NEPC report, using the same data (expanded to include all NYC charters, and cleaned&merged with additional measures) and same methods.  I have stayed with my original expenditure measure for BOE schools (defended here) and have used Annual Financial Report (nor IRS 990) data for charters.

Figure 3 shows that in New York City, charters are generally outspending traditional public schools serving similar student populations. KIPP and Uncommon schools are outspending BOE schools by over  $4,000 and $3,500 respectively and Harlem Childrens Zone schools are spending similarly (and that’s not even counting all of the additional money flowing to/through the parent organization. it’s just the annual financial report data!).

Figure 3. New York City Spending

Figure 4 shows more of a mixed bag in Houston. Non-major-CMO charters spend less than district schools. KIPP’s elementary schools spend less, but not consistently/significantly so (some do). Cosmos/Harmony schools also spend less, but through different means.  Others are indistinguishable, with some network schools spending more and others less (some Yes Prep schools outspend districts schools serving similar students).

Figure 4. Houston Spending

Have High Average Outcomes?

The following several graphs explore ht distribution of spending and outcomes for individual schools. Note that these graphs don’t include all of the other stuff that might need to be included to parse whether spending differences actually help produce outcome differences. That’s not the point here. Rather, these are just descriptive graphs of the relative spending and outcomes of these schools – focusing on schools serving middle grades.

Figure 5 shows that each of the charters spends quite a bit more than otherwise similar district schools. Each charter also has higher average performance (except St. Hope) than district schools. But, as shown above, they also have less needy students.  In other words, no miracles by these measures. Higher average outcomes yes. Lower spending? NO. Same kids? NO!

Figure 5. New York City Average Outcomes

Figure 6 shows the average spending and outcomes for schools in Houston. Among KIPPs, all spend much more than district schools and two have higher than average outcomes and the other two have average outcomes. The Yes Prep school in the sample spends more and has higher than average outcomes. Meanwhile, some other charters spend less and do less well, and one spends less and has somewhat higher than average outcomes. Notably, however, the population characteristics of these schools were also mixed. It may be the case that these KIPP schools have more needy populations than the average district school, and on average are doing average to better. That’s not bad. However, we must acknowledge they are doing this at a much higher price! Similar kids? Mixed (more low income, fewer ELL or special ed). Better outcomes? Also mixed, but okay. Less money? No, actually more… a lot more.

Figure 6. Houston Average Outcomes (standardized)

Have Greater Gains?

Okay, here’s one last shot. Let’s look at achievement gains instead of just level of performance. I’ve constructed school level average gains for NYC schools by aggregating the teacher value added data for teachers in each school (weighted for the number of students who contribute to their scores in English Language Arts or Math). In other words, in this analysis a school is only as good as its teachers (consistent with reformy wisdom) and, for that matter, the children served by (linked in data records to) those teachers.

Figure 7 again shows that in New York City, charters tend to significantly outspend district schools with similar populations – well except Equality charter which is somewhat closer. On average, the average gains are indeed higher in these higher spending charters – actually moving upward in sort of a pattern. But remember, the peer groups in these schools also aren’t particularly comparable. KIPP AMP and Brooklyn prospect, however, don’t do so hot.  But, if there’s any case to be made here with these charters, resources just might matter. Not the same kids. More money. Some reasonable outcomes.

Clearly, some deeper investigation is warranted. But, in each case there are also district schools, including lower spending district schools that outperform most of the charter schools.

Figure 7. New York City Value-Added

Finally, we’ve got Figure 8, showing the distribution of school level value added ratings from the FAST TEXAS system. Here, the KIPPs in particular are more of a mixed bag. Some have higher and others lower value added.  Note that in Texas these “progress” metrics seem to be associated with student characteristics.  All of the KIPP schools and the Yes Prep school spend more than district schools. Clearly, some deeper investigation is warranted. But, in each case there are also district schools, including lower spending district schools that outperform all of the charter schools.

Figure 8. Houston Value-Added

As a bonus, I also have this slide on class sizes (8th grade math) for NYC schools serving 8th grade. I’ve related class size to spending here, showing that each of these higher spending charter chains (except for Democracy Prep) seem to be leveraging at least some of that funding to provide much lower class sizes.

Bonus Slide: Class Size and Relative Spending in NYC

Closing Thoughts

In closing, we all really need to step away from the misguided logic I laid out in the beginning of this post – that “charter” in and of itself is meaningful  and that the only answers for the future of education must be found among successful charter schools – especially miracle charters under the watchful eye of Superguy. If there’s anything we know by now about Superguy is that he’s got some pretty nice financial backing!

Superschools & Miracle-Guy… uh… wait… Superguy and Miracle Schools, if they do exist, are a rarity. Further, if they do exist, they are as likely if not more likely (by sheer numbers) to be traditional public schools and not charter schools. But regardless of the governance of the schools, when looking for ideas (not “solutions”) for how to improve educational opportunities, we should be focused on the following:

1. Attempting to learn what we can from all types of schools and not predetermining that we should look at successful “charter” schools. This is especially true since charters are proportionately no more successful than other types of schools and, given that they are smaller in total numbers, they are in total, less numerous among all successful schools.
2. Being really, really careful about parsing out reasons for success before declaring schools to be miracles (and we should avoid the whole notion of “miracle schools”). We must look closely at population characteristics and population dynamics (mobility/attrition/neighborhood changes).
3. When we do find schools – charter or other – that appear to be making unexpected gains, we should explore what it is that they are doing. We should explore their resource use. We should explore the strategies they employ and we should figure out not just what it is costing them to adopt these approaches but what it would cost to adopt these approaches in other settings and more broadly.

Finally, publicness and true public access matters. While we may analyze and compare schools more thoughtfully regardless of their governance. I would now argue that when we consider the policy path forward we should actually give serious consideration to their governance.

All else equal, I am increasingly uneasy with the notion of creating larger numbers and shares of schools that are LIMITED PUBLIC ACCESS as I described in a previous post on charter schools. Intended or not, this is what has become of large segments of charter schooling.

I am concerned with the effect of expanded limited public access schools on those truly public schools around them. Intentional or not, this is how charter expansion seems to be playing out.

I am equally if not more concerned with the idea of shifting larger shares of children into schools where those children and their parents may forgo their constitutional and statutory protections, except where explicitly laid out in state charter statutes. Intended or not in the letter of state charter statutes, it is the charter operators themselves who invariably invoke their “private” status when defending the stifling of teachers’ free expression, or teachers’ claims seeking damages (under federal law) for mistreatment by a state actor, or students/parents claims regarding strict enforcement of discipline codes.

The maintenance of constitutional protections and true public access – non-exclusion – MUST play a significant role in the determination of the path forward. We should not be too quick to trade constitutional protections to employee or student free speech, privacy rights, protections from unreasonable searches and various statutory rights for a few additional points on state assessments or for a few dollars cut from school spending.

That said, the figures laid out above suggest that we likely aren’t even getting systematic or sizable bang for the buck when/if we do trade these constitutional protections. So perhaps that point is moot.

Related Resources

Baker, B.D., Libby, K., & Wiley, K. (2012). Spending by the Major Charter Management Organizations: Comparing charter school and local public district financial resources in New York, Ohio, and Texas. Boulder, CO: National Education Policy Center. Retrieved [date] fromhttp://nepc.colorado.edu/publication/spending-major-charter.

(Follow up: http://schoolfinance101.wordpress.com/2012/05/07/no-excuses-really-another-look-at-our-nepc-charter-spending-figures/)

Baker, B.D. & Ferris, R. (2011). Adding Up the Spending: Fiscal Disparities and Philanthropy among New York City Charter Schools. Boulder, CO: National Education Policy Center, 33. Retrieved April 24, 2012, from http://nepc.colorado.edu/publication/NYC-charter-disparities.