“When I use a word,” Humpty Dumpty said in rather a scornful tone, “it means just what I choose it to mean – nothing more nor less.”

“The question is,” said Alice, “whether you can make words mean so many different things.”

Since 1983’s Nation at Risk report, the dominant test score narrative has been frantic alarms interspersed with wailing and lamentations. No less than the imminent collapse of the nation was predicted (although we’ve managed to hang on for 32 years). But some have pointed out that our scores are quite good — except for our nationwide short-changing of our neediest populations.

Within the past two weeks National Assessment scores have been released in reading and mathematics. New England regional science results also became available. New independent research by Stanford professor Martin Carnoy and the Urban Institute’s Matt Chingos was also released examining how well our schools are scoring given various socio-economic factors.

We continue to score among the highest in the nation. Further, even after adjusting for Vermont’s advantageous demographics both independent researchers report Vermont as one of the highest scoring states. Additionally, our lead over the nation continues to accelerate, says Carnoy. We have the best graduation rate and we’re second highest in child well-being. An earlier American Institutes for Research report has Vermont among the highest test scorers in the world (tied for sixth in math and seventh in science).

But all is not rosy. For the nation as a whole, math scores went down and Vermont’s scores went with them. We don’t know if this is a one year fluke. New England regional science scores remained about the same but the percent showing “proficient” was 50 percent and below.

Yet, as Alice asked, how can words mean so many different things? How do we reconcile such contradictory findings? To get to the answers we have to consider differences in the tests, in the schools and whether outside-the-school events caused an entire nation to hiccup and dull-out their number two pencils.

Regarding the tests, one easily identified problem is that tests reporting the percentage “passing” are different from those with a scale score. Tests that report percent passing have to have a cut-score. If you are above the cut-score, you pass. Score below and you “fail.” The problem is in how these cut-scores are established. Unfortunately, they are not validated against a solid external criterion. They are basically the collected opinions of a selected group of people. Thus, the percent passing is a political decision based on the proportion the test-makers decide they want to fail.

This is how we end up with the absurdity of our science scores being among the highest in the world while half the student population is labeled as “failing.” Just as ridiculous is the claim that the tests tell us if a student, as young as the third grade, is on-track for being “college and career ready.”

The second place we must look is inside the schools. The decline in math scores is not because every teacher in the nation suddenly forgot how to teach. Three popular (but unproven) possibilities merit consideration: First, national educational spending dropped during and after the recession and has not recovered. Money does matter. (This is different from Vermont’s “cost per pupil” issue). Second, shifts in curriculum driven by the Common Core may have caught schools and students in transition. Third, after 15 years of at best modest growth, test based reform strategies like No Child Left Behind are not effective.

Finally, outside-the-school influences must be examined. The greatest predictor of test scores is socio-economic status. It accounts for between half and two-thirds of the variation in test scores. Economically, the bottom 40 percent has literally lost buying power while the top 1 percent has increased their wealth an astonishing 49 percent. At the same time, the ladder to the American Dream has been sawed off and social mobility has been hollowed out by the elimination of middle class jobs. This, in turn, means more homes with both parents working, greater financial stress, lessened access to high quality preschool for less affluent parents, and less parental time with children. In one year, Vermont children eligible for school lunch rose from 40.7 percent to 42.5 percent.

Certainly factors such as test design and school curriculum are vital. The inability of test-based reforms such as NCLB to move the needle is also important. But along with these factors, we must address the combination of tax, social and economic inequalities that define the health and the promise of our society.

And in answer to the meaning of the test scores, we must remember Alice’s rejoinder “The question is, whether you can make words mean so many different things.” Maybe the meaning of the words is that the schools and scores are also the reflection of the health of our society.

Published November 2, 2015 on VTDigger.org.