The U.S. News & World Report’s 2024 college rankings came out last week, as my colleague Nick Anderson notes, with a somewhat revamped methodology that does not escape what has been growing criticism of the enterprise.

Education Secretary Miguel Cardona captured some of the criticism last year when he said that any college ranking system that values wealth, reputation and exclusivity more than economic mobility and return on investment is “a joke.”

He didn’t mention U.S. News, though it was obvious that its college ranking was a primary target of his. But, as Harry Feder explains in this post, the annual undergraduate college rankings are just part of a larger exercise of ranking and sorting schools by the magazine in ways that critics say hurt authentic teaching and learning. Feder is the executive director the nonprofit, nonpartisan Center for Fair and Open Testing, known as FairTest, which advocates for the end of the abuse and misuse of standardized testing.

By Harry Feder

U.S. News released its latest version of the college rankings last week, and for the 2024 edition, the magazine modestly altered its formula in response to widespread criticisms and the refusal of some institutions to submit data.

The new algorithm tries to reward schools that make greater efforts at providing greater access by dropping the categories of class size and alumni giving (signs of wealth) and adding graduation rates of first-generation college students. As a result, some lower-profile public universities rose in the rankings. However, peer evaluation surveys remain a fifth of the rankings score, keeping the scores anchored in reputation, historic status and the sometimes not so subtle campaigning process by some schools.

Many colleges have criticized the ranking from a variety of perspectives over the years. Several have stopped participating in the rankings altogether, although U.S. News continues to rank them. One of the first undergraduate colleges to leave was Reed College in 1995, but few followed, though Colorado College did this year, with its president, L. Song Richardson, saying that even with methodology changes, the magazine “privileges criteria that are antithetical to our values and our aspirational goals.”

Though fewer than 10 undergraduate colleges have pulled out of the U.S. News rankings, several dozen laws schools have, a revolt that began when Yale Law School announced in 2022 that it was no longer participating in the beauty pageant. More than 40 laws schools followed suit, as well as some medical schools, including Harvard’s, early this year.

As problematic as the college rankings remain for a whole host of reasons, including methodological whimsy, the U.S. News rankings of high schools present a whole other host of problems. Standardized test scores make up no more than 5 percent of the undergraduate college algorithm (they only count for anything when a school has enough “usable” ACT/SAT scores) — but they are the driving force of the high school and K-8 school rankings.

By reducing lots of different school quality factors to a single number, all U.S. News rankings oversimplify academic quality. Because U.S. News seeks to rank in volume with insufficient investigative resources, its criteria fail to capture the nuances of the complex institutions. And the differences in school rank are not based on genuinely statistically significant discrepancies.

The U.S. News rankings are problematic in two ways: rankings incentivize behavior that runs contrary to the educational and social purpose of schooling and the benefit of students, and they perpetuate the very societal inequities and prejudices that we like to think systems of education should work to eradicate.

Virtually every single metric in the high school ranking is based on a standardized test. Fifty percent of the ranking is state assessment proficiency in math, reading and science. Thirty percent is “college readiness” or the number of 12th-graders having taken at least one Advanced Placement (AP) or International Baccalaureate (IB) course and earning a score of 3 or higher on the AP exam or 4 or higher on the IB. Another 10 percent purports to measure “college curricular breadth,” based on students who take four or more AP or IB exams that earn qualifying credit. Only 10 percent of the ranking is based on the school’s four-year graduation rate, the only non-test measure.

Standardized test scores, and the rankings that use them, strongly correlate to parental wealth and education and thus measure socio-economic status more than anything else. The inherent biases in these tests also tend to discriminate against Black and Hispanic students as well as English Language Learners. The ratings thus wind up being signals for socioeconomic and racial residential patterns and reinforce and ossify segregation and inequality. School rankings and scores are reflected in real estate prices.

The principal mechanism of funding schools in America is property taxes which are based on the market value of property. Without changing the way education is funded, rich school districts stay rich and under resourced ones remain so. Inequality is perpetuated over time.

Schools and districts are harmed educationally by the incentives created by the rankings machine. A community that cares about its rankings and its property values will pressure the school to do whatever it takes to achieve and maintain high test scores. Teaching to those tests becomes the educational objective. Particularly in under-resourced schools and communities, students get test prep above and beyond rich, engaging and culturally responsive curriculum and pedagogy.

Many standardized tests used in K-12 classrooms are poor measures of some of the most important academic and social skills we need young people to develop: critical thinking, the ability to collaborate with peers, oral and written expression, research skills, civic engagement, problem solving abilities and creative capacity.

The high school rankings focus on two types of standardized tests: tests mandated by federal accountability laws and AP exams. Because AP exams are such a key factor in the U.S. News rankings — schools with more AP classes and students taking AP exams rank higher — schools are incentivized to get students to take as many AP exams as possible. Schools will have students take the classes and sit for the exams whether it’s a good idea or not.

Using AP in this way turns out to be poor educational practice — as author Annie Abrams explains here — detracting from deeper learning, and perverting the original idea of the AP program. Students looking to gain admission to competitive colleges feel the pressure to take the maximum possible number of AP exams, mental health, deeper learning, or real academic interest be damned.

The equity issue is plain. Offering an AP class section in preparation for the test costs a school about $5,000 in math and humanities and $10,000 in the sciences. Taking the test now costs nearly a hundred dollars each. The school also must have the trained faculty to offer the course. Not surprisingly, students of color and poorer students are disproportionately underrepresented in the AP program. Black and Hispanic students also take the exam at lower rates after enrolling in an AP course than their White and Asian peers. And the schools they attend are penalized in the rankings, casting a pall on what otherwise might be a successful stable community.

Issues of race and class permeate K-12 school rankings. According to a study done by the education publication Chalkbeat, the more Black and Hispanic students enrolled in a school and the lower the parental income the lower the school is rated in GreatSchools listings (a competitor to U.S. News).

What is really needed are superior metrics and dashboards of information on school quality so parents and students can make genuinely informed choices and so that schools are incentivized to engage in practices other than test score consciousness and AP course overload. Dashboards should try to reflect the totality of what goes on in a school: Do students feel cared for? What is the teacher retention rate? Are parents satisfied with the school? What resources are devoted to college counseling? What is the arts programming? How well does the school do with students with special needs? How many books do students read in a year? How’s the school lunch? Can kids learn how to code?

By reducing school information to test scores, the complexity, richness and true quality of an institution is completely lost. And there is no incentive to improve in ways that genuinely matter.

School character and quality, like the character and quality of an individual, cannot and should not be boiled down to a single metric. Especially standardized test scores.