The University of Arkansas’ Department of Education Reform (EDRE) wins our Grand Prize for two reports. The first, Charter School Funding: Inequity Expands, argues that charter schools are underfunded. The second, The Productivity of Public Charter Schools, argues that despite being underfunded, charter schools still manage to produce better outcomes than district schools. The reports are impressive indeed—as long as we’re collectively willing to overlook the researchers’ bungling of their calculations of both the numerator and the denominator of their equation.

The charter school funding report was reviewed by Rutgers University school finance professor Bruce Baker. He noted the report authors’ misunderstanding of intergovernmental fiscal relationships, and he explained how this misunderstanding produced an erroneous assignment of “revenues” between charters and district schools. A district’s expenditure is frequently a charter’s revenue, since charter funding is often received by a pass-through from district funding. Thus, the EDRE report doubles up on the assignment of revenue to the public school districts: their own plus the charters’. The Arkansas authors also fail to take into account the fact that districts often retain responsibility for direct provision of services (such as transportation) to charter school students. Perhaps it should come as no surprise that the report suffers from vague documentation of its research methods and data sources. For example, it treats “all revenues” (not defined) as expenditures (which they assuredly are not). When numbers were not handy, report authors drew on murky “additional data sources.”

A productivity calculation looks at results per expense, so the enigmatic revenue calculation of the first report is used by the EDRE authors as their denominator. Their numerator computation is not much better, since it’s cobbled together with entirely inappropriate comparisons of student population characteristics.

That second report was reviewed by University of Colorado Boulder research professor Gene Glass, who found that it used state average NAEP scores without bothering to consider the well-known population differences between charter schools and non-charter public schools on demographic variables such as poverty (free lunch eligibility) or special-needs status. As Glass asks in his review, “If one is calculating ‘bang for the buck,’ what is left if neither the bang nor the buck can be believed?”

For these stunningly incompetent analyses, the University of Arkansas’ Department of Education Reform has thoroughly earned the 2014 Bunkum Grand Prize for shoddy research.