UPDATE: We seem to go through this every time I do one of these...



What I show here are the grade-level enrollments each year as a cohort -- the "Class of '17," for example -- passes through a school. This is not student level data; so far as I know, neither NYSED not NJDOE publishes attrition, backfilling, and retention data.



Yes, it's possible students are retained, which could make underclass grades larger than upperclass grades. But that raises a series of issues by itself -- is a school keeping students for five years at extra expense to the taxpayers?



The data is what it is. But the burden of proof is not on me -- it's on those who makes claims about the effectiveness of certain schools. 







I promise I'll get back to Newark in a bit. But before I do, let's talk a bit about education journalism and The Gorilla Channel...



By now, you've probably heard about the internet prank pulled by cartoonist Ben Ward. Ward tweeted out a completely phony "excerpt" from Michael Wolff's new book about Donald Trump, recounting a made-up story about Trump watching videos of gorillas fighting up to 17 hours a day.



It appears that quite a few people fell for the prank and passed on the tweet in ways that made it seem that they thought the story was real. One interpretation of the incident is that too many of us fall too quickly for stories that we desperately want to be true, even if, on their face, they are implausible.*



For me, the timing of all this was fortuitous, because I had just sent out a tweet that captured a paragraph from a blog by Diane Ravitch that asked journalists to be more credulous when covering "miracle" school stories:

Ahem. My take? Journalists should always question stories that involve miraculous claims about test scores and graduation rates. Skepticism should be their default attitude towards claims that sound too good to be true. If at first they take the bait, they will tend to stop digging and become defensive. Those who take the bait will look foolish, and indeed they are. When a school makes outlandish claims about test scores, ask first who graded the tests. Then check the process for excluding and selecting students. Ask whether the school has the same proportion of students with disabilities and English learners as neighborhood schools. Dig deeper. Ask whether it accepts students with cognitive disabilities, or only those with mild learning disabilities. Keep digging. It has been my experience that behind every “miracle” school there is either fraud, dubious practices (e.g., credit recovery), or careful selection and exclusion of students.**

What Diane describes here is painfully common in education journalism: some school official or PR hack (these days, usually from a charter school) calls up the op-ed page editor at a news outlet and pitches a story about a school that "beats the odds." The school almost always has some data point they are using to sell their "success." But the editor or reporter almost never follow through and put that data into proper context.



What emerges is a story that feels right to folks with reformy predilections, but isn't based on a full and proper accounting of the facts. "Miracle" school stories are like The Gorilla Channel: because people want to believe them, they suspend critical thinking and accept them without subjecting them to appropriate scrutiny.



Which brings us to this recent op-ed from the NY Daily News. Apparently, no one at the News thinks they should check their op-ed writers' claims to see if they actually make any sense:

It’s college acceptance season, and if you’re a senior at one of Democracy Prep’s high schools in New York City chances are you are very happy: No charter network has demonstrated more success getting its students into and through college.

[...]

According to the network, last year 189 of the 195 seniors in its three high schools that had graduating classes went on to college. And although the sample size is small (the network has graduated fewer than 400 students), the network estimates that 80% of its graduates either are still in college or have graduated. 

[...]

Nearly all Democracy Prep graduates are from low-income families. Nationally, less than 20% of high school graduates below the median of household income receive bachelor's degrees within six years. 

Do you want this to be true? Would it make you feel better if you knew that children from low-income families, growing up in cities whose schools have been under-resourced for years, didn't suffer a penalty in college admissions? Would it make you feel better about our nation's appalling neglect of its most vulnerable children if you could point to a few schools that "beat the odds"?



We know there is systemic, structural inequality and racism in America. We know our schools reflect this, and that educational outcomes are influenced by these ugly societal realities. But here's a story that calls these truths into question. How much do you want to set aside your skepticism? Enough to keep you from examining the facts?



Ok, then...



According to Democracy Prep's website, the network runs five high schools in New York City:

 

• Democracy Prep Charter High School
• Democracy Prep Harlem High School
• Bronx Prep High School
• Democracy Prep Endurance High School
• Harlem Prep High School

I am using NY State Education Department data for this analysis. I'll start by aggregating these five schools and asking a basic question: How many students who start as freshmen at Democracy Prep stay until their senior year?



This is important: if a school sheds students as they move from year to year, it suggests that school isn't placing all of its students in college -- it's only placing those who stayed for the full four years. What does the data show?

 

Year after year, Democracy Prep's charter schools shed substantial numbers of students between their freshman and senior year.



Is a school really placing all of its students into college if it loses substantial numbers of those students before their senior year? Is it fair to make a comparison of college placement rates between Democracy Prep and other high schools enrolling large numbers of low-income students if:

1. Democracy Prep only enrolls students who apply to be there?
2. Many students leave Democracy Prep before their senior year?

Keep in mind that the author of this piece, Charles Sahm, claims that cohort attrition is not an issue at Democracy Prep:

Unlike other charters that don’t accept new students after a certain grade, Democracy Prep — which operates elementary, middle, and high schools — takes in new students whenever a new spot opens up, even in later grades. [emphasis mine]

Maybe they do -- but Democracy Prep does not come close to replacing all of the students it sheds from year to year.

 

Did anyone at the News think to check Sahm's claim to see if it was both accurate and put into proper context? Did they think they have an obligation to their readers to make sure that relevant facts were being presented? Did they stop to consider that anyone making a bold claim like Sahm's should be required to subject that claim to rigorous scrutiny?



Education journalism in this country is too often shaped by Gorilla Channel values: because newspapers editors and publishers desperately want to believe certain things, they are willing to suspend critical thinking and merely report whatever they are fed.



How desperately do you want to believe in The Gorilla Channel? How desperately do you want to believe in "miracle" schools? Are you willing to have a good-faith argument about education policy?



Or is that too much to ask?

 

Another "miracle school" op-ed, courtesy of The Gorilla Channel!

ADDING: Sahm notes that Democracy Prep runs schools in other states:

Democracy Prep’s impact is not limited to New York. The network’s high school in Camden, N.J., is graduating its first class this year. All 34 of this year’s seniors have already received at least one college acceptance. This is revolutionary change for a city that five years ago only had three students graduate high school “college ready,” according to the College Board.

As luck would have it, I happen to have some familiarity with NJ education data...



Freedom Prep, the Democracy Prep affiliate in Camden, NJ, has not yet graduated a class. But we can see the cohort sizes up until last year.

 

This year's seniors at Freedom Prep were in a class of 78 freshman. By the fall of their junior year, they were down to 33 students. 58% of the freshmen of the Class of 2018 at Freedom Prep had left by their junior year.



Shouldn't this be discussed?





ADDING MORE: Gary Rubinstein calls out the Gorilla Channel values of the press's coverage of Success Academy. It ain't pretty...





* OK, look: I will be the first to say Donald Trump is unfit for office. But come on...



** Emphasis mine; lightly edited for clarity.