William Mathis is managing director of the National Education Policy Center and Vice-President of the Vermont State Board of Education. He sent this report to the blog exclusively. Hanushek is a strong believer in testing, accountability, and school choice, as well as annually firing the bottom 5% of teachers, as identified by their students’ test scores. 

Dr. Hanushek Again Imagines Saving Trillions of Dollars

William J. Mathis

In a recent brief published by the American Enterprise Institute, economist Eric Hanushek announces that $76 trillion (that’s a “T”) dollars will be generated or saved by adopting unspecified educational reforms.[i] That’s almost a four-fold increase! The closest explanation of these mysterious school improvements is that they are “incentives” for “teachers and leaders.” In a longer technical version of the brief, the authors dismiss the need for more specifics stating “the precise source of the given improvements is not important.”[ii](p.466)

Hanushek is the Paul and Jean Hanna Senior Fellow at the conservative Hoover Institution of Stanford University, best known for his frequent testimony on behalf of defendants in school-funding litigation. But his work also includes analyses of teacher effectiveness, among other topics.

The argument made in the new AEI brief is one he and his colleagues have made repeatedly over the years. It goes like this: If we imagine that standardized test scores go up a certain amount, this will add trillions to the GDP. Alas, this argument is little more than a numerological exercise in assumptions, multiplication and extrapolation.  The astonishing increased dollar amounts are the result of “new estimates of … human capital stock” and assumptions about growth, which are then extrapolated to the nation. It assumes that NAEP test scores are a valid measure of workforce quality which will, in turn, drive the economic gains. As noted, this improved workforce is developed through unspecified “significant changes in school policies.”

The AEI brief does not present the actual methods or results, but some of these are found in the original technical paper, which itself circles back to the authors’ other work. In the end, the analyses fail in predictable ways:

• They oversimplify the complex relationship between education and national wealth,
• They are based on correlations which are interpreted as causal,
• The statistical approach exaggerates the results, and
• Workforce projections do not support this static scenario.

Oversimplifying – Education and wealth are certainly connected. But assuming education has this pervasive power over the economy ignores the complex set of interactive factors affecting the economy. This includes infrastructure, transportation, investments, political instability, climate, poverty, social changes and macroeconomic supports.[iii] In fact, two-thirds of the variance in test scores is attributable to outside of school influences.[iv]

Correlations are interpreted as causation – “Gains from school activities” certainly sounds causal. More specifically, the long paper flatly states the “… gains in annual GDP [are] due to educational reforms.” (p.470). But then, they back away and state causality is “challenging.” (p.479)

The core assumption of the piece is illustrated in a scatterplot of test scores and state economic growth (Figure 1, reproduced below). This includes an “estimated impact” of education on a state’s wealth. Little or no consideration is given to more important variables that might be the causes of the limited correlation.

Statistical Exaggerations – Although a best fit line (regression) is super-imposed on the scatterplot, the pattern is more suggestive of a shot-gun blast than an illustration of a strong relationship. Going back to the original study, we find that test scores explain a mere 23% of the variance in economic growth – and this may be largely due to the aforementioned “third variables.” This weak relationship is far from being as strong as touted. Furthermore, achievement test scores are collapsed into “state aggregate scores.” When scores are collapsed in this manner, the effect is to exaggerate a correlation.

Mystery Methods – Beyond one short note below the study’s Figure 1, the AEI paper does not share how these numbers were derived. How these were calculated for each state goes unexplained. Improving teaching and teacher leaders is put forth as the leading reform, but this also is not defended or explained. Neither socio-economic factors nor adequate funding are addressed, although a vast and relevant literature points to their importance. Yet, improvements are claimed to be “enormous” and states should be “willing to make substantial changes.” But how these strong but unknown recommendations are derived is not explained. However, the long paper does present a rich trove of irrelevant statistical exotica.

Workforce Extrapolations – The brief misses the mark with its implicit assumption that the work force and economic needs will remain static and can be easily extrapolated. In an age of emerging artificial intelligence and with virtually every job being transformed in the next half-century, it seems a long reach to assume that the form, function and role of education (and the economic drivers) will remain unchanged.

[1]  Hanushek, Eric A. (May 2018). Every State’s Economic Future Lies with School Reform. Washington, DC: American Enterprise Institute. Retrieved May 29, 2018 from https://www.aei.org/wp-content/uploads/2018/05/Every-States-Economic-Future-Lies-with-School-Reform.pdf? FRPc013V1FcL1JnZHg4V0Vsd253Z0kzUWsxVjc3dGhUSGVzRURhWUZORyttNDlQdWR2aVgifQ%3D%3D

[1] Eric A. Hanushek, Ruhose, J. & Woessmann, L. (Winter 2017) “Economic Gains from Educational Reform by US States,” Journal of Human Capital 11, no. 4 (447-86). Retrieved June 1, 2018 from http://hanushek.stanford.edu/sites/default/files/publications/Hanushek%2BRuhose%2BWoessmann%202017%20JHC%2011%284%29_0.pdf

[1] Sala-I-Marten, X., et al. (2014). The Global Competitiveness Index. Retrieved May 29, 2018 from http://www3.weforum.org/docs/GCR2014-15/GCR_Chapter1.1_2014-15.pdf

[1] Rothstein, R. (2016). In Mathis and Trujillo, Learning from the Federal Market Based Reforms. Charlotte, N.C. : Information Age Publishing. p. 432.

[1] Hanushek, E (Summer 2011), “Valuing Teachers; How much is a good teacher worth?” Education Next, 11(3). Retrieved May 29, 2018 from http://hanushek.stanford.edu/publications/valuing-teachers-how-much-good-teacher-worth