For all my years of blogging about education, the most important part of any post is pursuing the right questions.

My first investigative post back in 2011 was inspired by listening to an Arne Duncan keynote speech in which he boasted that as a result of school closing that he authorized, the school that replaced the closed school got 100% of their 107 graduating seniors into college. It all sounded good until I researched the question “How big was that cohort when they were in 9th grade?” Throughout the years, most of my blog post have been based on asking questions like that.

Success Academy is the largest charter network in New York City. With 40 schools and 20,000 students, Success Academy is known for its high 3-8 standardized test scores and their rigid rules. Success Academy also celebrates the annual 100% college acceptance rate among its graduates.

Success Academy is a K-12 program and, until recently, the only time that you could enter the school was either in kindergarten, 1st, 2nd, or 3rd grade. The size of the first graduating cohort in 2018 was 16 students. The answer to the natural question of how many students started that cohort originally was 73 which meant that approximately 25% of the students who started with Success Academy eventually graduated from there. I say ‘approximately’ because it isn’t fully accurate to just divide 16/73=22% and conclude that 78% of the cohort left the school for one reason or another. Not counted in the 16 is the students who were still in the school but had been left back one or more years. It seems that about 6 more of the students from that cohort graduated a year later so maybe the true number is 22/73=30% is more accurate. But there’s another factor that, until now, has been impossible to factor in. Some of those 22 students are students who transferred into the school after the first year so you would have to subtract those students from the 22 to get the actual attrition rate. The only way to get that kind of data is to do a FOIL request which is exactly what I did.

Success Academy had 315 Kindergarteners in 2008. The graduating class of 2021 had 110 students. Without this new data, it would seem that their persistence rate is about 35%. But this new data I received shows that only 69 of the graduating class had started with the school as kindergarteners. So a more accurate estimate is 69/315=22% which is a little lower than the 25% I had originally estimated.

It is also interesting that 41/110=37% of the graduating class were from the backfills even though the backfills were from a pool of about 100 students. So about 41% of the backfills graduated vs 22% of the original cohort. A reason for this discrepancy could be explained by the way that Success Academy manipulates their backfill students to guarantee that the backfilled students are ‘better’ than the students they replaced. As I reported previously, lower performing students applying to be backfill students are often told that they have to repeat the grade they just graduated from which surely discourages some of them from accepting their backfill offer while higher performing students are not required to repeat the grade.

So like the famed Greek ‘ship of Theseus’ that had it’s planks gradually replaced until it couldn’t be considered the same ship eventually, the cohort of students who graduate Success Academy are very different than the cohort that started it.