In October, the U.S. General Accountability Office (GAO) released a report titled “Charter Schools That Received Federal Funding to Open or Expand Were Generally Less Likely to Close Than Other Similar Charter Schools” in response to a congressional request. The report looked at data about the federal Charter School Program, which over several decades has awarded billions of dollars in grants for the expansion or opening of charters. These schools are publicly funded but privately operated, often with minimal or no oversight from a governmental agency. The GAO said in part:

The Department of Education awards Charter Schools Program (CSP) grants to help open new charter schools or replicate and expand high-quality charter schools, among other things. While few charter schools closed overall, charter schools that received CSP awards closed at lower rates than similar charter schools that did not receive an award between fiscal years 2006 and 2020. GAO’s analysis found, for example, that within five years after receiving CSP awards, CSP-recipient charters schools were about 1.5 times less likely to close than similar non-CSP charter schools — with an estimated 1.4 percent and 2.3 percent closing, respectively. Within 12 years of receiving CSP grants, the same pattern generally held. The pattern also generally held for CSP-recipient charter schools regardless of the schools’ grade level, locale, student body racial and ethnic composition, or percentage of students receiving free or reduced-price lunch.

This post, written by Carol Burris, an award-winning former New York high school principal and now executive director of the advocacy group called Network for Public Education, raises questions about the report, saying that the GAO “used outdated charter school status data as the basis of their descriptive analysis.” She explains below how she came to that conclusion. Burris has written previously on the charter school program on this blog (for example, here and here), and in the following piece she takes issue with some of the GAO’s data and report results. The Network for Public Education is an alliance of organizations that advocates for the improvement of public education and sees charter schools as part of a movement to privatize public education.

The GAO denied that it used outdated data and said it stands by the report. It said that it needs “to use rigorous methodologies that are acceptable to social scientists and statisticians and can withstand scrutiny.” You can see its full response at the end of the piece.

The Department of Education was also asked for a comment and provided a short one that not directly address the GAO report or Burris’s critique. It said in an email: “Our administration is committed to supporting high-quality public charter schools, as reflected in the president’s budget. And we’re committed to accountability, transparency and fiscal responsibility in the federal charter school program, as reflected in our regulations.”

Burris said her data shows significant undercounting by the GAO of charter schools that closed after receiving federal grants from the Charter School Program — either through state governments or from the Education Department. She said she shared her data with the GAO on numerous occasions.

After repeated scandals in the charter school sector and negative fiscal impacts on public school districts from charter expansion, the Biden administration this year made changes to the Charter School Program in an effort to stop waste and fraud and bring more transparency to charter school operations.

In September, the U.S. Education Department’s Office of Inspector General released an audit of the nearly 30-year-old federal Charter School Programs that found, among other things, that charter school networks and for-profit charter management organizations did not open anywhere near the number of charters they promised to open with federal funding. Previous investigations by an education advocacy group, the Network for Public Education, which opposes the growth of charter schools, had found similar problems. (You can read my stories about their “Asleep at the Wheel” reports here and here.)

By Carol Burris

Congress last year directed the GAO to investigate the controversial federal Charter Schools Program (CSP), which was the subject of regulatory reform by the Biden administration this year. In a 2021 appropriations bill, the House Committee on Appropriation said:

The Committee requests GAO to provide a report to the Committees on Appropriations on the Department’s oversight over CSP and whether the program is being implemented effectively among grantees and subgrantees. The report should include an analysis of CSP grant amounts over time that supported charter schools, with a particular focus on schools that eventually closed or received funds but never opened; the relationships between charter schools supported by CSP grants and charter management organizations; and an analysis of enrollment patterns at these schools, especially for students with disabilities. The report should examine ways to improve the Department’s oversight of CSP as well as make recommendations on potential legislative changes to the program that would reduce the potential for mismanagement and ineffective operations.

The GAO report published in October does not address all of Congress’s mandate and, according to my research conducted over several months, severely undercounts the number of closed CSP schools and the federal dollars spent on them. In addition, that error has a ripple effect on findings throughout the report. What follows explains what went wrong, and the facts that back up these conclusions.

GAO’s numbers don’t add up

The published report, which covered only a small part of the congressional investigatory request, examined three programs, which they refer to as the State Educational Agencies/State Entities Awards, the Charter Management Organizations (CMO) Awards, and the Non-State Educational Agencies/Developers (Developers) Awards. The report contains a descriptive analysis of grants to schools that closed or never opened and a comparative probability analysis of grant recipients (new schools only) closing during their first 12 years. The comparative probability analysis, which became the headline for the report, was not part of the congressional request. Its findings are misinterpreted in the headline of the report.

This post, however, focuses on the requested descriptive analysis, which reported the present status (open, closed, future, will not open) of CSP awardee schools and how much was spent on those that never opened or closed. Its source was a data set given to the GAO by the U.S. Department of Education. That data set includes program information, school names, award years and amounts, identifying details, and a status for each grantee school — open, closed, opening in the future, will not open, or undetermined (as indicated by a blank) when their grant is complete.

In 2019, the department published a detailed data set of CSP awards, which you can find on the department’s website here. Most of that data set, specifically awards from 2006 through 2018, is a subset of the data set given to the GAO. The data set provided to the GAO also includes the 2019 and 2020 awards, however, we estimate that upward of 80 percent of the grantee information is in the public data set.

Let’s begin with a few examples of awardee schools and their status in the 2019 data set to understand why the report got it wrong.

Path Academy Charter School in Connecticut was a school that received a grant directly from the department. According to the 2019 data set, it received $585,800 in a three-year grant from 2013 to 2015. The data set reports the school’s status as open, but Path Academy closed in 2018 after the state discovered that the school and its charter management organization, Our Piece of the Pie, defrauded “the state of nearly $1.6 million, billing the state for 128 phantom students, operating unauthorized schools, and tolerating excessive absenteeism.”

Spirit Prep was a proposed “blended” school powered by the for-profit K12 (now Stride) online programs. It received a grant for over $186,000 in 2011 to plan for its opening. Although K12 announced in April of 2012 that Spirit Prep would open that fall, by July, the New Jersey Department of Education decided that the school would not open and denied its charter. In 2019, the department still had it listed as a “future” school with a note that it would open in 2012.

Tallulah Charter School, a Louisiana 2013 grantee, closed in 2017 following a cheating scandal. Its status is listed in the data set as open.

Hope Academy, a 2008-2010 grantee that received more than a half-million dollars, shut down in 2014 and was later sued by the state of Missouri for $3.7 million after “an audit found inflated attendance numbers.” Again, its CSP status was listed as open in 2019.

These are not isolated examples. They are representative of the hundreds of such cases that we found. Why do there appear to be so many errors?

The answer is that once the grant is finished (most end within three or fewer years), the department says it no longer checks to see if they are open. Therefore, the status of the school is frozen in time in the data set. A school open when the grant was complete may be shuttered today. The department requires that state entity, charter management organizations and developer grantees report twice a year on the operational status of all CPS-funded schools — but only for active and open grants.

This also explains why the Department of Education cautiously reports numbers of closed CSP schools using the term “closed prematurely.”

But the GAO did not check on the current status of schools, with the exception of the 189 schools that had no status in the data set. This is explained in Appendix I on pages 22 and 23 and was communicated to me in an email on Oct. 27 from GAO Assistant Director Sherri Doughty.

Recall that the GAO’s congressional mandate was “to report on CSP grants, with a particular focus on charter schools that eventually closed or never opened” (emphasis added). By accepting the department’s status in the majority of cases, it was using data that had not been updated in years, with the exception of 189 of 6,023 awards. Yet in the report, the GAO reports closures as current as of May 2022. Footnote 11 on page 11 says that the GAO defined “open” as currently open schools.

Despite my sending extensive file after file of correct information, their response was, “we stand by our report.”

Now, I will describe what they got wrong.

Extensive underreporting of CSP awardee closures

For the Network for Public Education’s analysis, we used the public 2019 CSP data set, which is a subset of what the GAO received. The vast majority (exceeding 80 percent) of the CSP awards from 2006 forward are in the data set, which covers 13 of the 15 years examined by the GAO.

Using the procedure outlined below, NPE’s Marla Kilfoyle and I identified the extent to which the GAO underestimated the number of closed and never opened schools, which were the categories of interest to Congress.

1. We isolated those awards in the 2019 data set made in 2006 and beyond, eliminating all awards made before 2006.
2. For all charter school awards with an NCES number (91.2 percent of all awards), we checked the school status against the 2020-2021 Common Core of Data (CCD). We marked charter schools as closed if they were no longer listed in the CCD, or if they converted to public schools while retaining the same NCES number. If a charter remained a charter with the same NCES number but changed its name, that school was marked open. In some states, including California, we double-checked with the state database. [NCES numbers are the unique 12-digit school identifier found in the Common Core of Data of the National Center for Education Statistics (NCES). We used the charter school filter in the CCD database to include awards that went to charter schools that closed as a charter and became public schools and to identify public schools that took CSP money but never converted to a charter school.]
3. If a public school received an award to convert to a charter school but did not, we marked it as “will not open.” If schools were listed as future schools in the data set that ended in 2018 but could still not be found in the CCD, we checked outside sources and, if not found, marked it “will not open.”
4. For the remaining 8.8 percent of schools, we accepted the school status as reported in the 2019 data set, knowing that would result in an underreporting of closed and never opened charter schools and an inflated number of open and future schools. We, therefore, erred on the side of caution.

Grantee closures

Let’s start with the smallest of the three programs, the Non-State Educational Agencies (SEA)/Developers awards, which I will refer to as non-SEA awards. These awards are given directly to charter schools by the Department of Education.

According to the GAO, the department gave out 235 non-SEA awards between 2006 and 2020. The 2019 data set, from 2006 on, contains 178 of those awards. According to Table 5 of the GAO report, only six went to schools that have closed, and four went to schools that never opened, resulting in a closure rate of 3 percent and a never-opened rate of 2 percent.

Using the CCD and additional outside sources to determine the status of schools, we found 29 — not 6 — schools that received a CSP award between 2006 and 2018 that had closed. Here we provide the names, date of grant, dates regarding the school’s closing, news stories about the closure, and other verification of closure.

Some charters closed due to low enrollment or poor test scores. Others closed, as confirmed by linked news stories, due to fraud.

We also identified 13 — not four — non-SEA grant schools that never opened between 2006 and May 2022.

Even if all of the 57 awards given after 2018 went to schools that opened and thrived (which is highly unlikely), closure rates would be 12.3 percent, and the never opened rate would be 5.5 percent of the non-SEA awardees, not 3 percent, and 2 percent.

SEA/SE grantee award closures and never-opened schools

The underreporting was even more dramatic when it came to the oldest and largest of the three CSP programs (SEA/SE).

According to the GAO, the CSP (SEA/SE) program gave 4,616 school awards totaling nearly $2 billion between 2006 and 2020. The 2019 data set identifies 4,351 SEA awards as sub-grants between 2006 and 2018. Almost all (3,992) have an NCES number associated with the school.

Within the data set, there is some duplication of schools. To catch those duplications, we identified and reported the number of unique closed or never opened schools. If we had reported by award, the number would be substantially higher. The GAO report is fuzzy in its tables and narrative, sometimes referring to schools and at other times to awards. It is possible for schools, especially longtime open schools, to receive more than one award; therefore, if the GAO counted awards, not schools, its “open school” number is inflated by more than error.

If the charter school did not have an NCES number in the data set, we again accepted the status listed by the department in 2019. As stated above, this likely results in an underreporting of closures.

GAO states in Table 2 that 429 SEA/SE awards went to now-closed charter schools — a number quite similar to the 2019 CSP data set non-updated number (409). However, we found that more than twice as many, 951 closed charter schools, received one or more awards. In addition, while the GAO reported that 209 schools never opened, we identified 230. These numbers do not include closed and unopened schools given grants after 2018. The total number is higher than what we report; it cannot go lower.

Note that we did not analyze the closures of charter schools that received Charter Management Organizations (CMO) awards since the department only required CMOs to report their schools beginning in 2012. The report lists 37 percent of that CSP CMO-grant funding going to “future schools.”

Our complete analysis is available upon request. It was sent to the GAO and the department along with a tool developed by data expert Ryan Pfleger that allows one to examine the history of schools by enrollment and status across the years of the CCD. I received an email acknowledgment and thank you from a representative of the Department of Education. I received no response from the GAO.

The CCD can be an imperfect source and may have generated minor errors in our final numbers. Nevertheless, it would have provided a far more accurate accounting of “schools that eventually closed” than the outdated status in the data set of the department they were asked to audit.

The ripple effect

The error described above directly affects the number of charter schools listed as open, closed, future, and will not open. It also affects the calculation of the total taxpayer dollars that have been wasted on CSP charter schools. For example, if more than twice the number of charter schools that received CSP grants closed, the GAO report’s estimation of $152 million spent on closed and never opened SEA/SE schools during those years is only capturing less than half of that cost since more funds went to closed schools than schools that never opened.

The state-specific numbers set forth on pages 13-15 of the report similarly need correction. Some of the states identified as the biggest wasters in the report’s Figure 15 may not deserve that identification. Other states may earn the dubious honor of being in the chart.

What now?

It is difficult to track charter school closures. Some schools close as charters and become public schools. We have seen schools switch between charter and public several times. At other times, a school shuts, and a new management organization takes it over. Sometimes the school’s name, staff, and students are different; sometimes not. Charter schools merge. In some states, information is easy to find; in others, information is obscure. It doesn’t have to be this way; states and the federal Charter School Program can demand better record-keeping and reporting.

The GAO’s descriptive analysis needs to be checked, verifying whether a school is currently open using the CCD. Claims regarding closed and open schools in their report need to be revised so that it is clear those are only closures during the active years of the grants. The stakes are even higher, however, for families. The closure of any school, whether public, charter, or private, is a painful and disruptive event in a child’s life. Families deserve honest information regarding closure risk when they enroll their children in a charter school. It is time for the GAO to revise its report to Congress and the public.

* * *

This is the response from the GAO:

We need to use rigorous methodologies that are acceptable to social scientists and statisticians and can withstand scrutiny. Practically speaking, we cannot Google the status of 6,000 schools and call that proper research. When we spot checked some of what Ms. Burris cited, we came up with conflicting results. As with any methodology and any data set, ours had limitations and they were disclosed clearly in the report.

In addition, GAO is an independent agency. We do work for Congress, but they do not dictate our research objectives, methodologies, or scope of work. GAO determined that the best way to meet Congress’s needs in this case was to conduct a descriptive analysis, which examines trends and relationships, and to pair that with a much more sophisticated model with rigorous controls in place. This was done to properly examine underlying issue at hand: the effectiveness of CSP awards. We laid out this approach to the relevant Congressional stakeholders prior to the work beginning, and they determined that it met their needs. And then it was laid out in our report as well.

We know critics who do not like our message will cherry pick at different statistics. But the message is based on a sound analysis and we stand by it.

Here is Burris’s response:

The GAO used outdated charter school status data as the basis of their descriptive analysis. The use of that data was confirmed in an email sent to me by the GAO and in the appendix of the report. The rationale for not using the Common Core of Data rather than the data provided by the Department they were auditing was illogical, especially given that they used the Common Core of Data for what they referred to as their “more rigorous model.”

The charter school status data they used is not updated once a grant is closed. This was confirmed in an email from a Department of Education spokesperson to Ms. Strauss. Therefore, when the GAO report states that its information is current as of May 2022, it is providing false information to both Congress and the public.

One does not need to “google” schools. The GAO is well aware that this is not the methodology I used. If their spot check resulted in conflicting results, I invite them to send those examples to me.