Here is the second part of an examination of U.S. News & World Report’s annual ranking of high schools. It was written by Kevin Welner, director of the National Education Policy Center at the University of Colorado Boulder. In theprevious post and in this one, Welner examines the process by which the magazine ranked the schools and declared them the best in the country. Welner is an attorney and a professor education policy at the UC Boulder. He is also the co-founder of the Schools of Opportunity project, an effort to identify and highlight high schools that use research-based practices to ensure that all students have rich opportunities to succeed.

A version of this appeared on The Conversation, and I asked U.S. News to comment. The magazine took issue with Welner’s argument in a statement, which you can find at the end of this post.

By Kevin Welner

PART TWO

I have recently been taking a closer look at the U.S. News and World Report ranking of “best high schools,” the nation’s most influential high school recognition program. In addition to surprising instances of carelessness, I discovered that the publication had deleted evidence concerning the earlier mistakes, and I found a troubling pattern whereby the rankings, which are supposed to recognize quality schooling, actually reward elite enrollment instead.

The earlier piece [below this one] points out specific technical problems with the rankings. In this second piece, I explore more fundamental problems with the rankings that would remain in place even if the technical problems were fixed. I look at the results of the U.S. News ratings process, and I question whether it yields rankings that meaningfully identify “best” high schools.

Recognizing Great High Schools—Is That What These Ratings Do?

As disappointing and troubling as the above-described problems are, a second set of problems looms even larger and affects schools in every state: how the rankings penalize schools that do not serve advantaged communities and/or do not enroll a selected group of students.

According to U.S. News, the rankings are based on two “key principles,” the first of which is: “that a great high school must serve all of its students well, not just those who are college bound.” Yet is this what the U.S. News ranking process actually does?

This year, the nation’s top-ranked high schools are: (1) Dallas’ “School for the Talented and Gifted,” a magnet school; and (2) BASIS Scottsdale (AZ), a charter school. The Dallas school has an application process that begins with a GPA screen and a test score screen, then the applicant student must “design and carry out a school-related or other project that demonstrates extended effort and creativity,” and then the applicant student must attend an “application session.” At this session, the applicant student must complete a timed (1 ½ hours) hand-written, prompted essay; must complete a “30-minute timed logic and reasoning activity;” and finally is given a 15-minute scored interview. This highly competitive process yields an entering class of 65 students.

The Scottsdale charter school is not selective in the same way; admission is by lottery. But the selectivity is arguably just as potent as with the Dallas school. Parents arewarned in no uncertain terms that only the most gifted and committed students will survive. Just a handful of students are admitted after the early grades (e.g., a parent was told that there would be a total of four slots open for entering eighth graders). Moreover, as described by the mother of a student in another BASIS charter, the warnings appear to understate the extremity of the BASIS approach.

These top-ranked schools, and the Dallas school in particular, may indeed provide great learning opportunities, but they hardly exemplify the spirit behind the supposed key principle for U.S. News: “a great high school must serve all of its students well, not just those who are college bound.”

There is an important lesson here about rankings in general—not just the U.S. News rankings. A disproportionate focus on outcomes will always reward those schools that excel at enrolling high-achievers. Perhaps this is more obvious when we look outside the school realm. For instance, University of Kentucky basketball coach John Calipari is such a fantastic recruiter that when looking at his success it’s almost impossible to disentangle his recruiting and coaching talents.

Yes, the school, teacher, or coach matters. Measured outcomes, which in the U.S. News case are primarily test scores, are attributable in part to the teaching taking place after the recruitment, even when a school brings in the best athletes or students. But here’s the secret: if that teaching is of low quality, the results will still look pretty good. The best estimates suggest that differences in what schools actually do will account for only about 20 percent of the variance in measured outcomes. The best way to produce high-scoring students is to enroll high-scoring students.

Top-ranked schools like those in Dallas and Scottsdale are expressly designed to serve only the most elite students. They cannot be scaled up and thus cannot serve as exemplars. This raises two questions. What does U.S. News intend to tell its readers when it places these schools on its pedestal? What does U.S. News intend to tell school leaders when it downgrades schools for enrolling a cross-section of America or for challenging non-elite students?

There is another option. Approaches for identifying high-quality schools like those used by the Coalition for Community Schools highlight schools and practices that can and should serve as exemplars to be replicated. In doing so, they provide an important alternative to what might best be called, “The Calipari Recruitment Awards” rather than the “Best High Schools” in America.

This brings us back to the American Indian Public High School, the top high school in California according to U.S. News. The formula used by the publication does not concern itself with what is actually happening at ranked schools; process isn’t important, only the numerical outcomes. Yet during the same year that this school was generating those great numbers (2012-2013), the school’s authorizer was attempting to revoke its charter. (Because California law makes test scores the primary factor for charter renewal decisions, the school brought a lawsuit and then reached a settlement allowing it to stay open, but with key problematic practices now changed.)

The authorizer’s serious concerns included the school’s disciplinary system, based on fear and humiliation. This system is what the school’s former leader, Ben Chavis, called “extra embarrassment,” meting out serious penalties for minor offenses, grounded in the “broken windows” theory of law enforcement. Chavis “once shaved the head of a student caught repeatedly stealing; some unruly students were forced to wear humiliating signs. And [he] often referred to black students as ‘darkies.’ ”

That, according to U.S. News, is the best high school in California!

Schools, if the research is to be heeded, should be recognized as great when they truly do what U.S. News purports to embrace as a key principal: “serve all of its students well, not just those who are college bound.” In fact, the best schools support and challenge all students with the broad expectation that those students can excel and can choose to become college bound. U.S. News has put a great deal of work into creating its brand and creating the infrastructure for telling parents across the nation about different schools. But those parents and the rest of us will be well-served by U.S. News only after it acknowledges and addresses its past mistakes, re-designing its approach to fit its rhetoric.

—-

I asked U.S. News to comment on the version of this that appeared inThe Conversation. Here’s the response:

Thank you for reaching out. Mr. Welner’s overall argument is incorrect. Schools that do better and are awarded medals in the U.S. News rankings are the ones that are doing more (better than the average in the state or way better than the average in the state) than what is expected given their level of poverty or proportion of economic disadvantaged. In other words, the U.S. News rankings do reward schools that push students to do more than is normally expected of them, in fact it’s a key premise of the rankings.