A group of charter school students rally in support of charter schools outside the Capitol in Albany, N.Y., on Tuesday, March 4, 2014.  (AP Photo/Tim Roske)

John Oliver, on his HBO show “Last Week Tonight” did a funny, biting segment on the charter schools, which educate a fraction of American school children — somewhere around 5 percent — but get a great deal more attention from policymakers then the numbers would predict.

Here’s a new look at charter schools from Carol Burris, a former New York high school principal who is now executive director of the nonprofit Network for Public Education. Burris was named the 2010 Educator of the Year by the School Administrators Association of New York State, and the same organization named her the New York State High School Principal of the Year in 2013. She has been chronicling botched school reform efforts in her state for years.

In a recent post, she explained why putting the word “public” in front of “charter school” — which are funded with tax dollars but sometimes considered private by courts — is “an affront” to people for whom public education is a mission. In this post, Burris looks at whether charter schools can properly be compared with district public schools — as they often are.

By Carol Burris

The New York Post, the charter school cheerleading paper of record, was giddy with delight when New York test scores came out.  Describing charter test scores as “sparkling,” the Post accused New York City Mayor Bill DeBlasio of “dissing” charters when he questioned their tactics. Jeremiah Kittredge, the chief executive officer of Families for Excellent Schools, went so far as to accuse the mayor of kicking little children “in the shins.”

When the mayor refused to back down and said charters have a history of excluding some kids, the Post editorial board jumped into the fray and said his claim was false.

Is the Post or the mayor correct? Are there differences between the kids who go to charters, and those who attend public schools in New York? Are charter scores that much better when you take those differences into account? And perhaps most important of all, are any score bumps worth the scams and scandals that have become a daily feature of charter school news?

New York City charter schools make up 81 percent of the charter schools in the state. Only 4 percent of New York’s charter students are English Language Learners, as compared with over three times as many — 13 percent — of the 3-8 students in New York City public schools. Fifteen percent of charter students in Grades 3-8 are students with disabilities, as compared with 22 percent of the students in New York City traditional public schools.  These differences in who attends charters are part of a national pattern.

Some of the gaps result from initial enrollment, and some are a result of charter attrition. Then there are differences in the degree of disability—a child with a mild learning disability is very different from one with severe autism or emotional problems. A 2013 study of Philadelphia schools by the Education Law Center provides important insights into distribution patterns by disability in charters— students with multiple disabilities, emotional disabilities, and autism were under-enrolled, with some at nearly half the expected rates.

English Language learners are also not a monolith. New arrivals with little if any fluency in English, have lower test scores than English Language Learners (ELLs) who are close to exiting services. About 15 percent of the 3-8 students who are ELLs in New York City schools have been in the United States for less than a year, as compared with less than 1 percent in New York charter schools[1].

Overall comparison of scores of New York City public schools and charter schools therefore, mask the impact on scores that results from different proportions and types of students. In New York City, students with disabilities and English Language Learners combined make up about 35 percent of all students; the groups combined for New York charters comprise 19 percent. So let’s see how those differences affect scores.

Before we begin, 2016 New York State tests scores are not comparable to last year’s—period. There was a new vendor, but even more importantly, the tests became untimed, there were fewer questions and there is even some evidence that the proficiency standard dropped. New York State Chancellor Betty Rosa, to her credit, made it very clear that claims of increases are not defensible given the changes to this year’s tests.

That being said, it is possible to compare the performance of one group to another on this year’s tests.

Overall, charter schools in the state (81 percent are in New York City) outperformed New York City public schools on the English Language Arts exam. The proficiency rate for charters was 40 percent, as compared to 36 percent for New York City traditional public schools. New York City charters performed slightly better than charters overall at 43 percent.

When we remove students with disabilities from the overall rates in order to mitigate the effects of differences in numbers of students and the severity of student disability, the charter advantage disappears. New York City district public schools have a proficiency rate of 46 percent and charters have a rate of 45 percent. When you then pull out the data for English language learners (New York City district public schools have more than three times as many), the New York City district public school test performance becomes stronger, and surpasses the proficiency rate for charter schools–50 percent vs. 46 percent.

In math, the “charter advantage” remains, but it substantially narrows.  The 9 percentage point spread (36 percent New York City district public schools vs. 45 percent charters) narrows to 7 (43 percent vs. 50 percent) when special education students are excluded. And when English Language learners are pulled, the charter school proficiency advantage narrows to 4 points (47 percent vs. 51 percent).

The above is not an exhaustive analysis of the data. Comparisons of subgroups always have limitations, but it does illustrate how differences in populations can have dramatic effects on test scores.

Far more sophisticated studies with better controls on variables, however, consistently show either no achievement benefits, or small achievement benefits for students in charter schools. The most recent study, entitled Charter Schools and Labor Market Outcomes, which used long-term data from Texas, found that on average, charter schools have no impact on student test scores and a negative impact on charter students’ future earnings.

What happens in the press is that the outlier charter, like New York’s Success Academy, with its tremendous resources and exclusionary practices, becomes the shining example, while the lackluster performance of charters overall is ignored.  What is also ignored is the exponential growth in scandals associated with charter schools across the country.

This week, the charter world had a wake-up call with the biting John Oliver critique of charters.  Nested in the humor was good research on the deep and abiding problems with charter corruption and charter closure.  Stories like those recounted by Oliver rarely move beyond the local press, so the extent of the problem is masked.

Just this week alone, over 500 charter school students in Livermore California fled to a local public school.  The Los Angeles Board of Education began taking steps to revoke the charter of the high-scoring El Camino Real Charter High School amid financial scandals.

The CEO of a Pennsylvania charter school resigned after an inappropriate advertisement to motivate students to enroll in her charter school appeared. Students in Detroit are worried they will not have a school to go to because their charter school issued a last-minute closure announcement.  And the founder of the Pennsylvania Cyber Charter school just pleaded guilty to siphoning off more than $8 million of school money to buy homes (one which cost nearly a million dollars), a plane and groceries.  All of the above occurred in the few days since John Oliver’s report.

In January 2016, four university researchers published a paper likening the proliferation of charters to the sub-prime mortgage crisis.  At the time, the paper received scant attention.  How ironic that it may be a late-night comedian who might finally alert the nation to the charter crisis. As Oliver noted, “the problem with letting the free market decide when it comes to kids is that kids change faster than the market. And by the time it’s obvious the school is failing, futures may have been ruined.”

The truth is, the deregulation that the high-scoring charter schools love so much, also produces dismal charter failures, taxpayer fleecing and fraud. And that, in the end, could cause the whole charter system to collapse.