New Orleans, seen from a ferry on the Mississippi River. (Matt McClain/The Washington Post)

School choice proponents love to talk about New Orleans after Hurricane Katrina ravaged the city in 2005 and the public school system was decimated. A collection of charter schools opened to replace the troubled traditional school district that had previously existed, and since then the city is often pointed to as a success for school choice and state takeovers of local schools.

Why is this effort called a success?

Standardized test scores are up from before the hurricane. But is the increase really impressive? The 2018 results for the Louisiana Educational Assessment Program exams found that only 26 percent in the Orleans Parish-Recovery School District had achieved “mastery” or above, less than the 34 percent state average.

(It is worth nothing that I don't think test scores should be viewed as significant measures of accomplishment, but school choice proponents do, so that is why they are being cited.)

So what is really going on in the schools of New Orleans? Are whatever improvements are being made happening for the reasons that charter school supporters say? Is it the “charterness” of the schools themselves or other factors that speak to traditional public schools as well?

That's what is discussed in this post by Carol Burris, a former New York high school principal who is executive director of the Network for Public Education, a nonprofit advocacy group.

Burris was named the 2010 Educator of the Year by the School Administrators Association of New York State, and in 2013, the National Association of Secondary School Principals named her the New York State High School Principal of the Year. Burris has been chronicling problems with modern school reform and school choice for years on this blog. She has previously written about problems with charter schools in California and a number of other states.

[What and who are fueling the movement to privatize public education — and why you should care]

By Carol Burris

New Orleans, post-Katrina, is undoubtedly the most cited example of the success of state takeovers, charters and choice.

Former education secretary Arne Duncan once said that Hurricane Katrina was the “best thing” that ever happened to education in the city (though he later apologized). The New York Times’s opinion columnist David Leonhardt recently praised the city in his series on New Orleans school reform. And the City Fund, led by Neerav Kingsland, the former chief executive of New Schools for New Orleans, uses New Orleans as a tool to pry open the coffers of philanthropy for its portfolio approach of school governance — one that would replace 30 percent to 50 percent of traditional public schools with charter schools in 40 cities.

When the data slides go up to pitch replacing public schools with “portfolios” and charter schools, you inevitably see research from the Education Research Alliance for New Orleans, led by Tulane University economics professor Doug Harris. Harris and his team have studied the post-Katrina school reforms of New Orleans for years. He and his colleague Matthew Larsen recently published the latest update on NOLA schools in a policy brief entitled, “What Effect Did the New Orleans School Reforms Have on Student Achievement, High School Graduation, and College Outcomes?” 

Their analysis found that test scores, high school graduation rates and college outcomes all improved for students who attended school in New Orleans post-Katrina. It is true that outcomes are up. The important question to ask is why the improvements occurred.

In order to answer that question, let’s first examine what the report’s author, Doug Harris, has to say. In his interview with the National Education Policy Center, Harris acknowledged several points that advocates of charter schools and so-called portfolio reform have largely ignored.

First, regarding the additional funding (almost $1,400 annually per student) flowing into the district, he noted, “It is difficult to estimate the role of funding or really any specific factor since this was a system-level change, involving several interconnected factors.” He added that without the spending increase, “The effects would almost certainly be smaller, but it’s hard to tell by how much."

Second, he explained that his sunny view of New Orleans' reforms cannot be extrapolated to other districts that might want to adopt the portfolio model or otherwise expand charters. “New Orleans,” he said, “was uniquely situated for these reforms to work. The district was extremely low-performing, and pretty much everyone agreed that some type of major change was in order. It’s easier to improve from such a low starting point. … I don’t think we can extrapolate New Orleans to most of the country. It’s more like a best-case scenario."

Bruce Baker is a professor of education Rutgers University whose areas of research include school finance and its effects on education outcomes. The Network for Public Education (NPE) asked Baker to review Harris’s policy brief. You can read his full review here.

Baker disagrees with the assumption that the portfolio-charter reform was the predominant cause of improvements. He identified two important factors downplayed by Harris and his team: the role of increased funding apart from the structural changes of school governance, and the significant reductions in the number of students who lived in extreme poverty, post-Katrina.

The role of increased funding post-Katrina

After the hurricane, as previously mentioned, per-pupil spending in New Orleans dramatically increased. In his review for NPE, Baker also pointed out that “instructional staffing expenses were held artificially low due to the influx of a relatively inexperienced teacher workforce, and changes to pensions and other benefits."

"It is likely that these expense reductions are not sustainable over time, meaning that total spending will either have to increase further to maintain the system, or that other expenses will need to be substantially reduced,” he said.

In other words, not only was there a substantial increase in spending (most of the increases went to supporting the high administrative and transportation costs of the new “choice system"), the district’s staffing costs after the storm were also unsustainably lower than before. During the years of the study, the district could therefore “buy more with less,” which would have the same effect as an additional funding increase.

So, does money matter in education? You bet it does, and economists have become quite skilled at identifying its impact.

For example, economists C. Kirabo Jackson, Rucker Johnson and Claudia Persico found that increases in school spending had substantial effects on long-term student outcomes. In their paper entitled “The Effects of School Spending on Educational and Economic Outcomes: Evidence from School Finance Reforms,” they found that 10 percent increases in spending increase graduation rates for students from low-income homes by 10 percent. A 2017 study by Christopher Candelaria and Kenneth Shores found a 10 percent increase in spending to be associated with an increase of more than 5 percent in the graduation rates of high-poverty districts.

According to Harris and Larsen, the 13 percent increase in per-pupil spending in New Orleans was associated with graduation rate increases between 4 percent to 9 percent — quite in line with what others have found in school districts across the United States, including those that have not experienced state takeovers and “portfolio” reforms.

In addition to national studies, state-level studies of California, Massachusetts and New York have found improved test scores, graduation rates and college-going rates due to increased spending as well.

Baker is right. Harris’s dismissal of the effects of the substantial increases in funding on the improved outcomes he found by casually referring to them as part of “the treatment,” is highly problematic.

Changes in the student population post-Katrina

Researchers also take into account demographic shifts in the students served by school systems because of the substantial role that household wealth plays in student success. Harris and Larsen claim demographic stability pre- and post-Katrina. Baker disagrees. He wrote:

When considering average shares of children who qualify for free or reduced priced lunch (under 185% income threshold for poverty), or other measures of central tendency (means, medians) for the city as a whole, this may appear true. But, there have indeed been substantial changes in the distribution of poverty across schools and neighborhoods and the concentration of extreme poverty in New Orleans.

Citing the work of a 2015 Brookings Institute study, he noted that “the share of the city’s poor residents living in neighborhoods of extreme poverty dropped from 39 percent in 2000 to 30 percent in 2009-13 (the latest small-area data available). This drop occurred at the same time that concentrated poverty rose dramatically in many major American cities, spurred by the Great Recession and slow recovery. As a result, whereas New Orleans ranked second among big U.S. cities in concentrated poverty prior to the storm, it ranked just 40th by 2009-13.”

Baker then explains why such a decrease could be at least partially responsible for the improved outcomes.

Additional concerns

As Baker pointed out, there are other important questions that go beyond what is easily measured: concerns of equity, structural inefficiencies, opportunities for waste and corruption, and the protection of the rights of the city’s most disadvantaged and vulnerable students.

Baker is certainly not the first to consider these questions. Louisiana teacher and researcher Mercedes Schneider explained the deep inequities that continue to exist in the system, writing:

It’s 2018, and black, public school students in New Orleans are still trying to have what white, public school students seem to come by in droves: access to schools deemed to be the preferable schools by the New Orleans community at large—with three of the preferable schools, having been able to elude until 2018 what is supposed to be a fair, open application process.

And in her recent book, “After the Education Wars,” Andrea Gabor, the Baruch College Bloomberg professor of business journalism, described New Orleans’s Recovery School District as a system of intense competition in which charter operators are “hoping to outperform the market for test scores, chasing a limited supply of philanthropic dollars."

"For children, there is the Darwinian game of musical chairs — with the weakest kids left out when the music stops and failing schools close, or when they were counseled out of schools that can’t, or won’t, deal with their problems,” she wrote.

The game

A few weeks ago, I was on a panel with Doug Harris discussing charter schools at an ideas festival called Kent Presents. Harris focused on the results of his study, without mentioning the substantial increases in per-pupil spending that he found, nor the role that the increase may have played in the improved scores and other outcomes. I brought that omission up during the follow-up.

One member of the audience was quite annoyed. “I don’t want to hear about poverty or more money. … Tell me what to do about kids that don’t behave and unaccountable teachers,” he said.

I was saddened, but not surprised. For several decades, reformers have told the wealthy and well-heeled that they can fix it all without having their taxes go up, or changing the economic inequities that brought them great wealth.

“None of these structural reforms cost public dollars,” Kingsland’s City Fund presentation reads. “Cities can increase the efficiency and equality of the system within existing budgets — with philanthropy supporting the transition costs.”

But that is not what Harris’s study shows. New Orleans gains came with a large price tag, and it was not a one-time expenditure.

The slide with the bar graph that shows the big jump in New Orleans’s per-pupil spending will always be the slide that conveniently gets dropped.