High school students in a science classroom. (iStock)

It’s that time of year where we get annual high school rankings from well-known sources. U.S. News & World Report does a prominent high school ranking, as does Jay Mathews, my esteemed colleague at The Washington Post.

I have published critiques of past rankings, questioning the methodology used by both to declare the “best” schools.  How is the “best” determined? Standardized test scores? They don’t reveal much more than the socioeconomic makeup of a school. Graduation rates? They are often suspect, in part because some schools report the number of seniors, fudging what might be a large dropout rate in earlier grades.

U.S. News recently issued its 2017 high school rankings. The top 10 looked like this:

1. 1. BASIS Scottsdale, Ariz.
   2. BASIS Tucson North, Ariz.
   3. BASIS Oro Valley, Ariz.
   4. School for the Talented and Gifted, Dallas
   5. BASIS Peoria, Ariz.
   6. Thomas Jefferson High School for Science and Technology, Alexandria, Va.
   7. BASIS Chandler, Ariz.
   8. Carnegie Vanguard High School, Houston
   9. School of Science and Engineering, Dallas
   10. Pacific Collegiate Charter, Santa Cruz, Calif.

Mathews, whose longtime “America’s Most Challenging High Schools” has been influential in what courses high schools offer, released his latest edition, and his top 10 looked like this:

1. BASIS Phoenix.
2. Mickey Leland College Prep, Houston.
3. IDEA Frontier College Preparatory, Brownsville, Tex.
4. IDEA Mission College Prep, Mission, Tex.
5. IDEA San Juan College Preparatory, San Juan, Tex.
6. IDEA San Benito College Prep, San Benito Tex.
7. Signature, Evansville, Ind.
8. BASIS Oro Valley, Ariz.
9. IDEA Quest College Preparatory, Edinburg, Tex.
10. BASIS Flagstaff, Flagstaff, Ariz.

[America’s Most Challenging High Schools] 

There are some crossovers in the list, but perhaps the most striking thing is how many charter schools — publicly funded but privately operated — are among the top.

Critics have jumped on the lists, noting the problems with comparing charters with traditional public schools. The highest-performing charter schools often have extremely high attrition rates and fewer students with disabilities and English-language learners than do traditional school districts.  Charter school management organizations often spend a good deal of money to market and recruit students, which traditional districts don’t do.

According to the U.S. News’s description of its methodology for ranking high schools, a four-step process was used. The first three looked at “math and reading parts of their state proficiency tests and their graduation rates as the benchmarks.” It says that graduation rates had to be 75 percent or greater to pass the threshold and that it was calculated on the basis of students entering ninth grade in the 2011-2012 school year.

Then the fourth step “assessed the degree to which schools prepare students for college-level work.” How was that done? By “using Advanced Placement test data as the benchmark for success.” No, the calculations did not include International Baccalaureate test data because the International Baccalaureate Organization decided to stop supplying the magazine with IB data. The AP content areas that were measured were English, Math & Computer Science, Sciences, World Languages & Culture, History and Social Sciences, Arts and AP Capstone.

How anybody could consider this a complete way of rating, much less ranking, a school is unclear.

Mathews uses AP data to make his rankings, too, but also uses other exams. He uses a simple index formula that is a ratio: “the number of Advanced Placement, International Baccalaureate and Advanced International Certificate of Education tests given at a school each year, divided by the number of seniors who graduated that year.”

Mathews argues that high schools challenging its students with this curriculum and then making sure that they take the tests — though they don’t have to get a passing grade — are “the most challenging.”

I have written repeatedly that judging a school by a single measure is at best incomplete. His rankings have, over years, led many schools who wanted to make a good showing in the “Challenge Index” to try to push students into advanced courses, which is a good thing for students who can do the work and not so great for those who can’t.

[America’s Most Challenging High Schools] 

Here’s a critique of Mathews’s index by Carol Burris, a former New York high school principal who is executive director of the nonprofit Network for Public Education. She has been chronicling problems with corporate school reform for years on this blog. Burris was named the 2010 Educator of the Year by the School Administrators Association of New York State. In 2013, the same organization named her the New York State High School Principal of the Year.

In this recent post, Burris wrote about charter schools in Arizona, a state with extremely lax laws governing charters, and critiqued the high-performing BASIS network, saying, “A close look at BASIS provides insight into how charter schools can cherry-pick students, despite open enrollment laws.  It also shows how through the use of management companies profits can be made — all hidden from public view.”

Mathews wrote a piece in support of BASIS in reaction to that post, “Should charter schools be allowed to set extremely tough standards?” Here’s Burris’s latest piece.

[What the public isn’t told about high-performing charter schools in Arizona] 

[Should charter schools be allowed to set extremely tough standards?] 

By Carol Burris

We may not know the origins of the universe or who placed the boulders of Stonehenge. The identity of Jack the Ripper eludes us, and whether or not Sasquatch exists remains subject to debate.

But rest easy — the superiority of charter high schools is now a settled question. Nina Rees, the president of the National Alliance for Charter Schools, and Howard Fuller of Marquette University present the proof in their Newsweek opinion piece, with the brassy title, “Proof Positive that Charter Schools are Better.”  It says:

“That’s it folks; the debate is over. Charter high schools are equal to or better than their traditional peers. That’s a fact.”

So where is the evidence to back that claim? The majority of studies comparing charter schools versus traditional public school indicate that overall achievement of charter schools is the same or worse than traditional public schools. Like public schools, charters vary in student outcomes.

The charter high school graduation rate is 70 percent, far below the “traditional” high school rate of 85 percent. The percentage of charter high schools that are low-graduation schools is 30 percent, compared to 7 percent of “traditional” high-schools. It seems to me that getting kids to the finish line would be a pretty important measure if you claim to be better than the rest.

So what is the breaking news on which Rees and Fuller base their claim? Their “proof positive” can be found in Jay Mathews’s America’s Most Challenging High Schoolslist. The evidence pops right out, according to the authors — “charter public schools filled out 9 of the top 10 spots.”  There you have it — case closed[1].

When Jay Matthews first published his Most Challenging High Schools list in 1998, I was a big fan. America’s high schools needed a kick in the pants when it came to academic tracking. High schools used test scores, recommendations and matrices to shut too many students out of  higher-level courses. Advanced Placement and International Baccalaureate courses were reserved for only the elite student.

It was overdue for high schools to open the gates. Teachers and administrators needed to allow more students to challenge themselves. This was the philosophy of Jaime Escalante, one of Jay Mathews’s heroes, who was a calculus teacher at the high-poverty Garfield High School in East Los Angeles and the subject of the 1988 film, “Stand and Deliver.”

And for a while, Jay’s list served that positive purpose — it provided recognition to schools that made Advanced Placement and International Baccalaureate classes available to more kids. Its rankings were, and still are, based on a simple formula — the number of tests divided by the number of members of the senior class. That sort of made sense when the list began. Most AP and IB tests were given in the senior year.

But elitism never dies an easy death. If more seniors were taking AP courses, there must be something special for the select kids who were the only ones previously allowed to take AP. So more schools began giving AP classes in the 11th grade. And then AP moved to the 10th and ninth grades, and even into some middle schools.[2]

The push did not end with the creation of elite underclassmen tracks. Charter schools that required passing more than 10 AP classes in order to graduate were created to attract the academically elite. Others required students to pass an AP test to get a high school diploma, in addition to taking a large number of AP classes beginning in middle school.

The list that sought to dismantle the exclusive AP track incentivized the creation of the exclusive AP charter or magnet school.  It should come as no surprise, then, that the sort and select machine, not equity, is now rewarded by Jay’s list.

Let’s take a look inside this year’s top three schools on Jay’s list.

#1 BASIS Phoenix:

The BASIS Phoenix graduating class of 2016 had 24 students — fewer than the average New York City kindergarten class. It began four years earlier with 43 ninth-graders. The drop from 43 to 24 represents an attrition rate of 44 percent. Jay’s list says that the school’s enrollment is 757 students, but that is deceiving because BASIS Phoenix is both a middle and high school. The entire high school population (which is what the list is about) in 2016 was only 199[3].

BASIS Phoenix does not have a free or reduced-price lunch program, and it does not provide transportation. It asks its parents for a $1,500 donation per child each year, along with hefty fees to participate in sports and extracurricular activities. In 2016, the school had so few students with disabilities, the state could not list the number without violating privacy — not even to give a total for the entire school. Thirty-three percent of its students were Asian American and 57 percent were white.  In Maricopa County, Arizona, where the school is located, 3 percent of the students are Asian American, and 41 percent are white. The majority of Maricopa County students are Latino, and 47 percent receive free or reduced-priced lunch.

Because Jay’s formula uses the senior class enrollment as the denominator to create the Challenge Index, schools with high attrition rates that give AP exams to underclassmen are rewarded. This results in BASIS Phoenix’s absurdly high Challenge Index of 26.250. If you used the original number of students who entered the high school as the denominator, the Index would drop to 14.65. Losing kids who can’t keep up has it rewards.

I asked Jay why he continues to use the size of the 12th-grade class as the denominator, knowing that sizable numbers of students leave some of his top schools. His rationale for ignoring attrition is that he does not want to exclude schools with high dropout rates. But I can’t find any such schools in his top 100, which is now filled with charters, magnets and private schools. Places such as Escalante’s Garfield High — with its senior class of 559 that includes 60 students who are still struggling to learn English — cannot possibly compete for a spot on the list with the elite 24 graduates of BASIS Phoenix.

#2 Mickey Leland High School

This school is a magnet school and the only non-charter in the top 10. Its 2016 graduating class was diverse and the majority received free or reduced priced lunches — all 11 of them[4]. That’s right: The #2 school on the list began with a class of 17 ninth-graders and graduated 11 students in 2016. That is fewer kids than in a special education self-contained classroom. Its attrition rate was 35 percent.

#3 IDEA Frontier

This school had a more reasonable 2016 graduating class — 104[5]. The attrition rate was over 10 percent. Nearly all of its students are Latino. The school reported a free or reduced-priced lunch rate of 87 percent.

IDEA has a Challenge Index of 19.59. That index means that the school gave 2,038 AP tests[6] to its roughly 400 students. And since class size is relatively stable across the four grades, an Index of 19.59 means that by the time kids graduate, the average student takes about 19 AP exams.

That seemed awfully high to me, so I called the school and asked about the scope and sequence of the curriculum — specifically what AP courses do students take? A guidance counselor at the school was both helpful and pleasant, and walked me through the IDEA Frontier program, grade by grade. The school is transitioning to IB, but she explained the AP program for the Class of 2016.

Freshmen take one AP course — Human Geography. Sophomores take AP World History. In Grade 11 students take four AP courses — English, U.S. History, AP Physics and AP Spanish. In Grade 12, they take between four and six APs.  The school requirement is 11 APs in all, which is what the counselor said most do.

Plugging in the numbers based on the 2016 enrollment, and assuming five APs in Grade 12, yields 1,125 tests, not 2,038. Even if every 11th and 12th grader took eight tests — meaning every period was a distinct AP class in Grades 11 and 12, that would produce 1,829 tests. That number assumes that the average student takes 18 tests over the course of four years, far above the counselor’s estimate.

It is possible that students are taking tests for classes in which they are not enrolled, or retaking AP tests; both would certainly pump the numbers for “the list.” It could be human error or perhaps the counselor was not as knowledgeable as she seemed.

The above highlights another problem with “the list.” Jay does not get his data and school enrollment numbers from objective sources. Rather he prefers that schools self-report.

Checking the numbers with the College Board, along with checking enrollment numbers on state websites, makes sense. Schools should explain their course of studies to ensure the tests are attached to courses the student is presently taking. It is supposed to be a measure of challenging curriculum, not test-taking.

Now let’s go back to where we started — Rees’s and Fuller’s  “Proof Positive that Charters Schools are Better.” The authors conclude their opinion piece by telling us to take “the lessons charters have taught us and apply them to all of our public schools, so that every kid has a chance to learn and succeed.”

What, then, are the lessons public schools should learn from the “top schools?”

Should our neighborhood schools follow the lead of the top charters and cater to the strivers and the gifted so those who cannot complete 11 AP courses, or pass an AP course, are forced to move out?

Should ranking lists call high schools “the best” when their program keeps teenagers with Down syndrome and serious learning disabilities out, or when they shed 10 percent or more of their students who cannot keep pace? Should we then have “default” public high schools where the students who can’t keep up are segregated from more academically able peers? If we continue down the path of unfettered choice with vouchers and boutique charters, that will surely be the outcome.

If, however, we believe that the good school equitably serves all children, there must be a balance between reasonable challenge and inclusivity. Asking all students, with the exception of students with the most challenging disabilities, to take an IB or AP course or two before graduation is an idea I support.

However, when we establish schools that create exclusivity by design, or by their unreasonably difficult graduation requirements, we are not furthering equity. And that results in lists more appropriate for Ripley’s Believe it or Not, than “best schools” lists.

Correction: An earlier version incorrectly said that the Most Challenging Schools index uses test scores. It uses number of tests taken in a ratio involving number of students — not the scores.

[1] Rees and Fuller also cite the presence of charters on U.S. News & World Report Best High Schools list as proof. That list also rewards schools based on many of the same assumptions about quality as Matthew’s list, producing an even more selective group of schools, including test-in schools. This ranking list has its own set of additional issues, including error that comes from a lack of understanding of state assessments.

[2] The International Baccalaureate, to its credit, only allows students to take IB courses in the 11th and 12th grades. Many of those courses are also two-year courses.

[3] Enrollment figures and demographics for BASIS Phoenix can be found here:http://www.azed.gov/research-evaluation/arizona-enrollment-figures/

[4] You can find enrollment numbers for Mickey Leland here: https://rptsvr1.tea.texas.gov/adhocrpt/Standard_Reports.html

[5] You can find enrollment numbers for IDEA Frontier here https://rptsvr1.tea.texas.gov/adhocrpt/Standard_Reports.html

[6] To arrive at the number of tests given multiply the number of students in the senior class (104) by the index (19.596)