Welcome to my world, ed policy folks: small impacts can be very meaningful, but primarily in the long term. One of the sadder bits of the post-Lowrey story Chetty, Friedman, and Rockoff (s)extrapolation is the attempt to magnify the scale of outcomes beyond what the research shows. The economists concluded that the impact of shifting a single year’s teaching up one standard deviation were observable but small improvements in several early-adult outcomes, including indirect measures of college attendance, income at age 28, and early parenthood. The conclusion that shifting up one standard deviation for a year meant a one-percent increase in late-20s income must not have seemed very meaningful to the authors, so they performed various gyrations to come up with a more headline-worthy figure. It’s still “approximately one percent.”¹

So, too, with President Obama’s argument in his State of the Union address that states should require school attendance until either high school graduation or age 18. As Sue Dynarski pointed out on Tuesday night on Twitter, some recent research by Philip Oreopoulos suggests that increasing the upper end of the compulsory-school range can improve early adult outcomes, including reducing unemployment. I can poke some holes in the methods of Oreopoulos,² but the scale seems about right to me: observable if small effects.

So what do we do with evidence of small effects? One frame is return on investment, which is what Hank Levin and Cecilia Rouse’s column in the New York Times presents today. They put the increase-the-compulsory-exposure-to-school proposal in a broader context with other ideas: quality early childhood education, small class sizes in primary grades, and higher teacher salaries. And then they pull magnified illustrations out as rhetoric:

When the costs of investment to produce a new graduate are taken into account, there is a return of $1.45 to $3.55 for every dollar of investment, depending upon the educational intervention strategy. Under this estimate, each new graduate confers a net benefit to taxpayers of about $127,000 over the graduate’s lifetime. This is a benefit to the public of nearly $90 billion for each year of success in reducing the number of high school dropouts by 700,000 [cutting the dropout rate in half] — or something close to $1 trillion after 11 years…. Some might argue that these estimates are too large, that the relationships among the time-tested interventions, high school graduation rates and adult outcomes have not been proved yet on a large scale. Those are important considerations, but the evidence cannot be denied: increased education does, indeed, improve skill levels and help individuals to lead healthier and more productive lives.

Levin and Rouse aren’t quite correct: the relationships among these issues are demonstrated, but not over the short term–they are evident over the first two-thirds of the twentieth century, as high school graduation became the majority, expected experience of teenagers.³And therein lies the rub: if you could guarantee that roughly 200% return on the investment, but you might see that benefit on a large scale in 20-40 years, how many policymakers would go for it?

Levin, Rouse, Chetty, Friedman, Rockoff, and many others are using what I think of as thePascal’s Wager wager: betting that a rhetorically magnified impact argument will carry the day.⁴ The problem is that this becomes translated into overpromising at the scale at which policymakers care: 2-4 years at most.

Is there a way to make policymakers invest in the long term? With the inability of federal Republican officeholders to move on transportation infrastructure spending in the past few years, out of a combination of Norquist Terror and Obama Spite, I’m not sure it’s possible any more.

Notes

1. If the CFR, Lowrey, Kristof, and Carey claims are irresponsible extrapolation, one could create an equally-valid argument that because the one-year influence of a 1SD change is about 1%, then schools don’t make much difference. Why that is irresponsible reading of the CFR paper is left as an exercise for the reader, but keep in mind the symmetry with the opposite claim… [↩]
2. As is usual with such analyses, the labor market isn’t static but most analyses of cross-sectional data assume a non-zero-sum demand for high school graduates. [↩]
3. The reasons why are undertheorized, despite the work of Claudia Goldin and others. I think Heckman is on the right track with his focus on noncognitive benefits of education, but there’s more going on. And that’s a topic for a different day. [↩]
4. It doesn’t matter that the economists are estimating finite effects: in each case the rhetorical argument is pretty close to Pascal’s infinite-effect-of-grace claim. [↩]

Related posts:

1. What “multiple measures” looks like in reality
2. The right kind of infection
3. Education history and school renewal