Autonomy in exchange for accountability. That’s been the value proposition offered by the charter school movement from its inception. Freed from the bureaucracy clogging the workings of traditional public schools embedded in dysfunctional school districts, charters would engage in bold, innovative practices that could serve as models for diverse communities of children. In exchange for this autonomy, charters would be held accountable by their authorizers for results. If they were unable to demonstrate their ability to meet their goals and objectives, their charters would not be renewed, and they’d go out of business.

The value proposition remains as controversial today as it was one and two decades ago, when charter schools were little more than blips on the educational landscape. Some studies have found that attending a charter school enhances student achievement, but there is also evidence that context matters, with charters performing no better, and sometimes worse, than traditional public schools in large swaths of the country. Charters have also struggled to shake the impression that they engage in “creaming,” enrolling and retaining students who present the least challenges to educate, and shunting the more challenging students—English language learners, and students with serious disabilities—to traditional public schools. In many communities, charter schools enroll lower concentrations of these challenging students than do the nearby traditional public schools.

The charter school movement got a boost with the election of President Barack Obama, who installed Secretary of Education Arne Duncan to preside over an initiative seeking to expand the number of charter schools across the country. Many states, wary of the implications for the public-education system of a rising number of privately managed schools, had set caps on the number of charter schools that could be authorized across the state. The Race to the Top competition, which dangled billions of dollars in front of states, defined charter expansion as central to statewide innovation, and bribed states to remove their caps on the number of charter schools.

In May 2010, New York’s legislators raced to pass laws that would make the state competitive for Race to the Top, lifting the cap on charter schools statewide from 200 to 460, and creating a new teacher and principal evaluation system based heavily on students’ performance on standardized tests. In lifting the cap, New York sought to ensure that charter schools would provide equal access to the students living in the communities in which the charters were to be located. The charter school law, therefore, created enrollment and retention targets for charter schools. To retain their charters, charter schools were to enroll students with disabilities, English language learners and students eligible for free or reduced-price lunches at rates that were comparable to the percentages of these students in the local community. Having enrolled such students, charters were also to retain them from year to year at a rate comparable to the retention rate in the local community. (“Retention” here means hanging on to a student from one year to the next, not the rate at which students might be retained in grade versus promoted to the next grade.)

When we use the expression “the devil is in the details,” sometimes it’s hard to discern who the devil is. New York’s legislators charged the New York State Board of Regents with prescribing the enrollment and retention targets for charter schools, and the Regents relied on the New York State Education Department (NYSED) to develop a methodology. NYSED’s Charter School Office collaborated with the State University of New York’s Charter Schools Institute to develop a methodology for the minimum enrollment and retention targets. Although the law was passed more than two years ago, there is now a race to implement it; the public comment period on the proposed methodology ends today, on May 29th, even though new material pertaining to the targets and their calculation was posted as recently as May 27th.

The NYSED provides spreadsheets that use data from 2010 and 2011 to illuminate the proposed methodology by showing potential enrollment and retention targets for both traditional public schools and charter schools across the state. I looked at the retention targets for New York City charter schools for Limited English Proficient (LEP) students. There are 94 charter schools in the sample spreadsheet—but only four of them have retention targets for LEP students. The proposed methodology exempts 90 of 94 charter schools from any accountability for retaining LEP students.

This is bad statistics and bad public policy. It’s bad statistics because the methodology proposed by the state relies on statistics that require large samples. A school is exempted from having a target if there are fewer than 10 LEP students in the school, among other criteria. But there are common statistical adjustments for estimating proportions in samples that have fewer than 30 students, or fewer than 10 in a given category.

It’s bad public policy because the method does nothing to hold the vast majority of New York City charter schools accountable for retaining their most challenging students. In the 2009-10 school year, New York City’s 94 charter schools enrolled 2,378 LEP students. The educational fate of the 1,960 LEP students enrolled in the 90 charter schools that lack a retention target is ignored by the proposed methodology. That hardly seems consistent with the intent of the legislation.

Autonomy in exchange for accountability—but who is holding the New York State Education Department accountable?