On the heels of the latest call by the NAACP for a charter school moratorium, there has been a media blitz started by The 74 about a report called ‘The Alumni’ in which they claim that charter school graduates go on to graduate college at three to five times the rate of low income students who do not attend charter schools.

Besides being reported in The 74, it has also appeared in The Wall Street Journal and The New York Daily News.

The 74 article is written by Richard Whitmire (as is The Daily News and The WSJ Op-Ed) who is known for his biography about Michelle Rhee (haven’t heard much about her lately) and also one about Rocketship Charters (haven’t heard much about them lately).

The summary of the report says that they have tracked the students at nine charter networks and found that graduates of those charters have between 25% and 50% of those students also graduate college.  Since a commonly quoted statistic is that only 9% of low income students graduate college, these networks seem to be getting between three and five times the rate of college completion.

The major flaw in this report — and they admit this in The 74, but not in The Daily News (The WSJ is behind a paywall, if someone can read it let me know if they address it there) — is that while the 9% statistic is for ALL students who enter schools, these 25% to 50% numbers are only for the students who complete 12th grade at the schools (KIPP is an exception, they use data from students who complete 8th grade — I’ll get to that later.)

In The 74 they actually do an entire other article explaining all the issues with the data that could cause the numbers to be inaccurate, which is something I appreciated, though of course the big takeaway will still be the three times to five times numbers that will be quoted, I’m sure, for the next decade by charter zealots.

Now there is really no way to verify these claims.  We don’t have a list of students who graduated and then another list of which ones graduated college.  But there are, thankfully, a few numbers that can be analyzed in the report.  I’ve done one network so far and hopefully will be able to do more another time.

Here is a quote from The Alumni about the Uncommon Schools network:

• Uncommon Schools: For the New York–based network, the only alumni who have reached the six-year mark graduated from North Star Academy Charter School in Newark. (The alumni from its Brooklyn high school just reached the four-year mark. Of the 142 North Star students who reached the six-year mark, 71 earned four-year degrees: a 50 percent success rate.

I went to the New Jersey data page where they have the databases for the enrollment at their schools throughout the years and downloaded all the data from the 2010-2011 school year down to the 1998-1999 data.

According to their data, there were eight graduating classes that have been out of high school for at least 6 years, the classes of 2011, 2010, 2009, 2008, 2007, 2006, 2005, and 2004.

The sizes of their graduating classes were, respectively, 24, 14, 36, 19, 27, 20, 23, and 19 which is a total of 182 students, not 142.  More importantly, going back more years I found that these 12th grade classes had lost about half their students from when they entered the school as 5th graders.  The 5th grade classes for those cohorts were about 40 each.  So if they really got 71 college graduates out of those 8 cohorts, it is not a rate of 71/142=50% but instead 71/320=22%.

Yes, I know that is still ‘double’ the expected 9%, but there are other factors that might make students who apply to charters have a higher rate.  Really we can’t be sure that there really were 71 graduates, but just taking them at their word for that, it still is dishonest to claim that it is a 50% college graduation rate.

Now the KIPP network claimed a 38% college graduation rate but they claimed that the other schools had inflated numbers because the other eight networks only counted students who completed 12th grade at their schools while KIPP counted students who completed 8th grade at their schools.

Here is a quote from the article about this:

KIPP is a fervent believer that college graduation cohort data should be tracked from ninth grade — not 12th grade, the starting point that the other charter networks included in this study use.

For students who attend KIPP middle schools, KIPP tracks them when they graduate from eighth grade to ensure they are kept track of, regardless of whether they go to a KIPP high school.

For students who go to non-KIPP middle schools and start attending KIPP as high schoolers, they track them when they start ninth grade.

The problem with starting in 12th grade, argues KIPP, is that it could tempt schools to push out weaker students during high school years, thus allowing the stronger students to boost the schools’ college-going and college-completion rates.

KIPP may be right. But in The Alumni, where KIPP is the only network that is currently tracking students from ninth grade, we have decided it is important to share cohort graduation rates that start in 12th grade. What’s key to this series is learning what works in boosting that college graduation rate — lessons that could be passed along to all schools, not just charters. Moving everyone to the gold standard is the next step.

KIPP is correct that schools that only count students who complete 12th grade will have inflated scores compared to KIPP that counts students who complete 8th grade.  But what KIPP doesn’t mention is that the fairest way to make a comparison to the 9% number is to start counting at 5th grade.  KIPP actually has a pretty big attrition between 5th and 8th grade so the true ‘gold standard’ is really not used by anyone.  All the numbers are inflated.  KIPPs might be inflated less than the others, but it still is so they can whine that the others are cheating worse than they are on this statistic, but they should admit that they are doing it too, though to a lesser degree.

There isn’t a lot of detailed data included in the report, just a summary of the main findings.  And I’m not so confident we can trust the data in there anyway, but it still is interesting to see how the numbers are inflated by ignoring attrition.