With the Oscar celebration next week, and the Emmys and Pulitzers on the way, the National Education Policy Center announces this year’s winner of its Bunkum Award. We invite you to enjoy our 10^th annual tongue-in-cheek salute to the most egregiously shoddy think tank reports.

It’s not easy to laugh when data are manipulated and made to fit foregone conclusions or when the research literature is misrepresented or ignored and low-quality or dishonest “evidence” has real impact on policy and on children. As best we can tell, polar bears aren’t laughing at reports from the American Petroleum Institute.

Yet “humor is one of the best ingredients of survival,” according to Aung San Suu Kyi—whose travails have been far weightier than ours. So we will persevere in our commitment to having a bit of fun each year with the evidentiary farce-lympics.

The Think Twice Think Tank Review Project arose as a response to the often-outsized policy influence of glossy, well-publicized reports that have not been vetted by peer-review. These reports regularly wrap themselves in the veneer of research, but they are frequently little more than propaganda masquerading as social science.

This year’s awards announcement, available on the NEPC website, is hosted by Dr. David Berliner, the Regents’ Professor Emeritus and former dean of the College of Education at Arizona State University. Berliner is a member of the National Academy of Education and the International Academy of Education, a Fellow of the Center for Advanced Study in the Behavioral Sciences, a past president of the American Educational Research Association, and a widely recognized scholar of educational psychology and policy.

The 2015 Bunkum Winner

This year’s winner is the National Alliance for Public Charter Schools, for Separating Fact from Fiction: What You Need to Know about Charter Schools. The National Alliance (NAPCS) describes itself as “the leading national nonprofit organization committed to advancing the charter school movement.” Separating Fact from Fiction is a fetching, sleek publication adorned with 15 charming photos of smiling children keeping watch over 21 easy-to-digest, alleged “myths” followed by responses that the report generously describes as “facts.” Yet Separating Fact from Fiction might more honestly be titled:

Playing 21 with a Stacked Deck
or
Blackjacked! 21 Attempts to Club Sound Policy.

Before turning to a small sampling of the report’s problems, however, we’ll offer a compliment: To the credit of the report’s authors, the 21 so-called myths do a good job covering many of the important issues raised by the rapid growth of the charter school sector. Alas, this comprehensive coverage is wasted. The framing of the myths is often so crude as to merely set up a straw man, and the presentation of facts in response to those myths is often outright misleading or unresponsive to the report’s own question. The report’s Blackjack-playing authors (who chose not to identify themselves) decided to bamboozle their readers using several celebrated techniques: the palmed card, the false cut, and the sucker’s bet.

The Palmed Card. The first myth listed in the report is “Charter schools are not public schools,” and it’s answered as follows: “As defined in federal and state law, charter schools are public schools.” Indeed, this is a technically true statement. A reader of the report, however, is distracted from the palmed card. The reader would never learn that charter schools live in a highly nuanced and ambiguous part-public and part-private niche. Charter schools receive public funding, students receive a no-tuition education, and charters are subject to certain non-discrimination and testing laws—so they’re public in some respects. But they’re private in other respects. As explained in the NEPC review (internal citations omitted):

• Most charter schools are governed by nonprofit boards. It is increasingly the case that charter school buildings are privately owned by the charter’s founders, by an affiliated private company, or by a private trust.
• In schools operated by private education management organizations (EMOs), the materials, furniture, and equipment in the schools are usually privately owned by the EMO and leased to the school.
• Except for a small number of states that require teachers to be employees of the charter school, it is common for teachers to be “private employees” of the EMO. 
• Although most charter schools have appointed nonprofit boards intended to represent the public (i.e., taxpayers’) interest, a growing portion of charter schools are operated by private EMOs, and key decisions are made at corporate headquarters, which are often out-of-state. [About half of the nation’s charter school students are enrolled in schools owned and operated by private EMOs.]
• Public schools, like other public entities, are subject to transparency laws. Charter schools and their private operators increasingly refuse to share information and data in response to public requests.

False Cuts. Other “myths” discussed in the report falsely suggest that proof is being offered when, in fact, all that’s offered is a bald and sweeping claim, a proclaimed truth or a glowing affirmation. These False Cuts never really rebut the criticisms of charter schools; rather, they simply assert that their self-described myths are not true. For example, when answering the charge of “less qualified teachers,” the report says, “charter leaders aim to hire talented, passionate, and qualified teachers who will boost student achievement and contribute to a thriving school culture.” Similarly, their defense of the “myth” that the charter movement is “anti-union” is that the schools are actually “pro-teacher.”

So they aim to hire good teachers, and they’re pro-teacher as well. One can only imagine how relieved the readers of this report were, particularly those who worried that the NAPCS would announce that charters were anti-teacher and, in fact, aimed to hire untalented, unmotivated and unqualified teachers.

In fact, the report as a whole suffers from evidence getting lost in the shuffle. Its use of the research literature borders on the cartoonish: in examining its 47 endnotes, our reviewers found exactly one article that had been published in a peer-reviewed journal. In contrast, almost two-thirds of the citations were to publications of advocacy organizations. Keeping in mind that charter schools have been as extensively studied as any policy over the past decade, these omissions are astonishing; a search in Google Scholar for “charter schools” yields 41,500 results. As the report has no methods section, the reader is left adrift as to why the report selected the biased set of documents it did and why it ignored a vast and easily accessible body of scholarly literature.

The Sucker’s Bet. Three of the report’s first four myths concern money: “Charter schools get more money than other public schools,” “Charter schools receive a disproportionate amount of private funds,” and “There is a lack of transparency around charter schools’ use of funds.” The gist of the authors’ arguments in response to these “myths” is that charter schools are responsible stewards of unfairly small allocations of public money.

They’re inducing readers to make a Sucker’s Bet—to go all-in because of the amazing efficiency and payoff that charters supposedly deliver. These readers are never told that the key research relied on for these claims had already been debunked. As our review noted, the lesser public funding received by charter schools “is largely explained by charter schools spending less on special education, student support services, transportation, and food services.” There’s nothing nefarious here, except perhaps some deceptive and incorrect calculations. Our reviewer explained:

When comparing public funding of charter schools with that of district schools, it is critical that the portion of ‘pass-through’ funds to charter schools from school districts be subtracted. Otherwise, the district revenues are erroneously and vastly inflated. For instance, if a public school district has the responsibility of providing transportation of charter school students, then the taxpayer funding for that transportation should be attributed to the charter schools, not the public school district. But sloppy calculations do not do this. … Nevertheless, Charter schools can receive … additional (categorical) funding if [they wish by, for example, serving] more children with moderate or severe disabilities and if they start offering programs such as vocational technical programs that would qualify them for targeted funding.

It’s one thing to count your money while sitting at the table—it’s quite another to miscount it and the money of the other player, which is essentially what the study did that the NAPCS relied on. It incorrectly attributed the provision of district-funded activities (e.g., transportation) to the charter, and it incorrectly attributed the revenues spent on these services to the district.

And so on it goes for 21 “myths.” The result is a series of rudderless assertions, insisting that there is nothing to be concerned about in the charter school world, except of course that charters are being unfairly treated. Yet as the NEPC review explains, the actual picture is much more nuanced. The charter sector includes many good people doing good things, many cases that are highly troubling, and a very real need for improved practice and regulation. With charter schools, just as with any policy issue, whitewashing the record does nothing to advance sound public policymaking or to help the charter sector move forward.

The National Alliance for Public Charter Schools earned this year’s Bunkum Award, as the worst example of educational policy research in a think tank report, in part because of the report’s sweep—because of the sheer acreage it unashamedly covers. In a repetitive fashion that fatigues the reader, it sets up and knocks down myth after myth, pretending that it is engaging in a truth-telling mission. This is a shame. Over the years, NAPCS has vacillated between being an honest purveyor of research evidence to being a blind advocate. This time, a useful report was not in the cards.

Watch the 2015 Bunkum Award video presentation, read the Bunkum-worthy report and the review, and learn about past Bunkum winners and the National Education Policy Center’s Think Twice think tank review project, all by going to
http://nepc.colorado.edu/think-tank/bunkum-awards/2015.