There is a crowded industry devoted to rating and ranking schools and states. The cool part about these ratings is that the imperious think tank raters get to assign a grade of F Double Minus to states and schools that have not adopted their group’s cherished reforms.

The competition is especially plentiful and fierce in this category, but Herbert Walberg and Mark Oestreich of the Heartland Institute vanquished all contenders with their 2010 State School Report Card. Our honorees ranked states (and the District of Columbia) on student achievement, low education expenditures, and “adherence to learning standards,” as well as a ranking based on an average of the first three. They created the indices and validated them by a rigorous examination of their preconceived proclivities and then highlighted the top- and lowest-performing states for each of the indices. Although the report explains how the indices were devised, it does not cite any research or rationale to support the methods, other than the data sources themselves plus reports from “several think tanks.” As the report acknowledges, the researchers also don’t even try to control for state variations in demographics or other factors. The report then assigns letter grades to each of the states (plus DC), with a forced distribution: 10 states are assigned A's, B's, C's, and D's, and 11 states must get F's. The beauty of this normative system is that it guarantees it will always be raining – there will always be failing states. Even setting aside problems with the criteria chosen, this seems a bit problematic. Anyway, after acknowledging that there is “very little empirical evidence” as to why some states do better than others, the authors stroll along to their conclusion that education quality is poor and that school choice is the obvious remedy. Assuming there was an acceptable method in this pile of subjective accretion, it is completely unconnected with these final recommendations.

The runner up in this category is a report from the Education Trust, called Stuck Schools, which suggests a framework for identifying chronically low-performing schools in need of turnaround. Among four main problems identified by the reviewer was the following:the norm-referenced methodology guarantees "failed" schools independent of any true performance or improvement level by the school. Sound familiar?