It’s about time we all suck it up and realize that the best of economic research on factors associated with test score gains not only can, but must absolutely drive the redesign of our obviously dreadful American public education system! [despite substantial evidence to the contrary!]

With that in mind, I have selectively mined some of my own favorite studies and summaries of studies in order to develop a framework for the absolutely awesomest school ever! I’ve chosen to focus on only economic studies of measurable stuff that is actually associated with measured test score gains. After all, that’s what matters – that’s all that matters!

Mind you that this school will be awesomest not merely in terms of overall effectiveness, but also in terms of bang for the buck, because I’m not messin’ around with expensive curriculum or elaborate facilities… or high priced consultants… or really expensive strategies like class size reduction.

I’ve chosen to avoid enrolling grades K-3 since the research is actually pretty strong that I should offer smaller class sizes in those grades. If I don’t have those grades, I guess I don’t have to worry about class size! Right?  In the absence of such clear research for grades 4 to 8 (or my choice to ignore that which really is relevant), I’ve decided that when it comes to class size, anything goes.

I’m goin’ for low hangin’ fruit here. Keepin it simple – with class sizes of 60 or so (since we know that doesn’t matter???) , running my school in a vacant lot and with absolutely no administration and/or supervision – since I’ve negated the need for the principal role in guiding high quality teacher selection by using an alternative, necessarily cost effective strategy!

So, here goes… Here’s my Econometric Academy Middle School (Grades 4 to 8).

Hire and keep only those teachers who have exactly 4 years of experience

First, and foremost, since the research on teacher experience and degree levels often shows that student value-added test scores tend to level off when teachers reach about the 4^th year of their experience, I see absolutely no need to have teachers on my staff with any more or less than 4 years experience, or with a salary of any greater than a 4^th year teacher with a bachelors degree might earn. Anything above and beyond this is simply inefficient. Paying a teacher more after the 4^th year is simply inefficient. Boosting 4^th year pay is also inefficient if I can simply, in perpetuity, employ only teachers with exactly 4 years experience.

Here’s a graph from a Calder Center report summarizing the student test score gains in relation to teacher experience.

http://www.caldercenter.org/UploadedPDF/1001455-impact-teacher-experience.pdf

Now, I’ve reviewed the various economic simulations that suggest that dismissing teachers on the basis of student value added test scores is a reasonable approach to, over time, increasing teacher quality. For my nifty new school, I choose to believe in their assumption that there will always exist a normally distributed flow of new applicants whose average quality is the same as the current pool of teachers.

My approach allows me not to even worry about selecting out the bottom 5 or 10% and replacing them with average teachers. Instead, I’m going for cost-effectiveness! You see, if the average teacher has already achieved their likely best value-added outcomes by year 4, then (accepting the current experience based pay system) at year 4 I’ve got teachers who are at their maximum productivity and the lowest wage – and I don’t have to ever worry about paying them more! That’s totally freakin’ awesome! I just have to make sure that every year, when I let my entire staff go, I get out there and find a totally new crop of teachers who have just completed their third year of teaching elsewhere – and are at least “average” among soon-to-be 4th year teachers at producing outcomes. Thus, every year, I will have teachers who have the average production of 4^th year teachers and the average wage of 4^th year teachers.

That is, they are necessarily better than average in terms of cost effectiveness.

This is a no brainer!

Implement carb loading on testing days, scaled up w/grade level (& in spring where fall-spring assessments are given)

Now, let’s shoot for some somewhat more obscure ideas… that have great potential to yield some nice marginal gains to tests scores on top of my already optimally staffed school. For my next few clever strategies, I turn to the work of David Figlio formerly of the University of Florida and currently at Northwestern (yes… this is a sarcastic post… but Figlio is a truly exceptional scholar… really clever guy… and one of the nicest people you could ever meet. Plus, he produces some really fun food-for-thought!).

Research from back in 2002 found that under Virginia’s accountability system, many school districts were adjusting their lunch menus to increase carb loading on SOL (uh… standards of learning) testing days. More importantly, David Figlio and colleagues found that it worked!

Using detailed daily school nutrition data from a random sample of Virginia school districts, we find that school districts having schools faced with potential sanctions under Virginia’s Standards of Learning (SOL) accountability system apparently respond by substantially increasing calories in their menus on testing days, while those without such immediate pressure do not change their menus. Suggestive evidence indicates that the school districts who do this the most experience the largest increases in pass rates.

Specifically, the authors note:

We observe that the estimated effect of calorie manipulation is positive across all five tests, and is statistically significant, despite the extremely small effective sample size, in the case of mathematics. (Figlio, food for thought)

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.159.8754&rep=rep1&type=pdf

Yeah… this is really low hanging fruit (perhaps quite literally) for my bang-for-the-buck econometric academy. All I have to do is carefully plan out my school menus to optimize their influence on student test scores. I might want to think carefully about how to play this right in a value-added structure. For example, if we have fall-spring assessments, do I carb load only in the spring?

(from the authors acknowledgement section)

The opinions expressed in this paper do not necessarily reflect those of their employers, funders, or young children, the latter of whom respectfully disagree with the authors’ derogatory characterization of “empty calories.” We, in turn, blame them for any remaining errors.

Rename all students prior to entry

Figlio has also produced an intriguing series of studies that consider how students’ names affect their behavior and performance in school. There’s some more low hanging fruit for making my school the best performing school ever with little additional costs! The policy implications are absolutely clear from the research – I must review the names of all incoming 4^th graders for two potential performance inhibiting characteristics. First, are their boys who have names that sound like girls names? Second, are there kids with either names that sound “black” or names that sound like they were given by less educated and/or lower class parents.  Next, I simply have to require that the parents of their kids change their names prior to starting 4^th grade. To aid these parents in making good name choices, I will have available a list of gender appropriate, Asian sounding names, because that too is backed by the research!

Here’s the research behind my brilliant, cost-effective proposal… and it is both statistically significant, and compelling!

Racially identifiable names:

The persistence of the Black-White test score gap, and its widening over the course of the school cycle, is an issue of significant public policy concern. This paper presents evidence that a portion of these patterns could be due to the names given particularly prevalently to Black children. Children with names associated with low socio-economic status, and to a limited degree, with “Blackness” per se, tend to score lower on their reading and mathematics tests, relative to their siblings with less race or class-identifiable names.

This hypothesis is also bolstered by the finding that the opposite set of results are observed in the instance of Asian families, for whom a racially-identifiable name may signal attributes that are perceived to be associated with success. Asian children with racially-identifiable names apparently face higher teacher expectations and also tend to score higher on examinations.

http://faculty.smu.edu/Millimet/classes/eco7321/papers/figlio.pdf

Boys with female sounding names:

 I find that, as suggested above, boys with female-sounding names tend to misbehave  disproportionately in sixth grade, as compared to other boys and to their previous (relative) behavior patterns. In addition, I find that behavior problems, instrumented with the distribution of boys’ names in the class, are associated with increased peer disciplinary problems and reduced peer test scores, indicating that disruptive behavior of students has negative ramifications for their peers.

http://www.aeaweb.org/assa/2005/0107_1430_1102.pdf

Have salaries based entirely – not just partially – on loss aversion tied to test score gains

Finally, what kind of econometric academy would I have if I didn’t totally buy into the most recent study on  loss aversion as a compensation strategy! Roland Fryer and colleagues have provided us a real gem here. Fryer and colleagues find:

In this paper, we demonstrate that exploiting the power of loss aversion—teachers are paid in advance and asked to give back the money if their students do not improve sufficiently—increases math test scores between 0.201 (0.076) and 0.398 (0.129) standard deviations.

http://scholar.harvard.edu/sites/scholar.iq.harvard.edu/files/fryer/files/enhancing_teacher_incentives.pdf

Now, I’m going all out with this one. Every teacher gets paid their full salary at the beginning of the year (I’ll have to use some kind of accounting trick to deal with the timing of my state aid and local transfer payments, or once I’m up and running, rely on the money I took back from the previous year teachers to pay those up front salaries the next year). If your kids’ scores don’t increase more than the average, you lose your whole salary (see, it’s all relative, I get half the salaries back every year no matter what!).

I can see how this strategy might create a divisive culture in some schools or might create animosity between teachers and administrators for teachers who repeatedly lose compensation – and may lose that compensation largely as a function of random error and/or omitted variables bias in the model designed to estimate their effectiveness. Yeah… I can see how taking teachers’ salaries away for factors that may be entirely outside their control could really piss them off, more than actually inspiring them to try to control these uncontrollable in a subsequent year.

But, my school is different. I really don’t have administration… because decisions are already made, by me, at a great distance. Besides, I don’t have to see any of my teachers the following year anyway because they all get dismissed every year, and a new crop of 4^th year teachers – equal to the previous – comes in (at least I think they will…). I just have to inspire them (scare the crap out of them) to kick some butt for that one year!

And that my friends, is the Econometric Academy of Achievement Test Excellence!

Closing thoughts

But seriously, much of the past week or so seems to have been dominated by discussions of Roland Fryer’s new NBER working paper indicating that while typical merit pay incentives don’t seem to influence student outcomes (by increasing teacher effort), when those incentives are paid up front, and taken back in response to lower performance, gains can be noticed. The buzz phrase (and theoretical framework) for the analysis is “loss aversion,” and a common assumption is that people may have more incentive to work harder if they fear losing something they already have, as opposed to gaining something they never had.

This stuff is fun to ponder (in a warped, academic sort of way), and potentially interesting as a research topic. But it’s all highly questionable in terms of usefulness for improving school quality (note that I said school quality, not test score gains!).

And that’s true of a lot of educational, psychological and econometric research related to schools. It’s especially true of any one of these branches of research in isolation!

The real key with most of these studies and others like them is to avoid the leap that these studies have immediate decisive policy implications – that they can and should be used to inform school reform – school redesign and state and federal education policy more broadly. Yes, each bit of information can advance our understanding. But, we must avoid the urge to assume that each new tidbit provides a new silver bullet answer and also negates all that we’ve learned previously.

Policymakers (and newspapers) want research with immediate and obvious policy implications. They want the silver bullet. They want the breakthrough that negates all previous understanding – that tells us why everything we’ve done to date is wrong and paints a clear path forward. Unfortunately, too many researchers feel compelled to play along.

Consider the great Chetty, Friedman and Rockoff one great teacher can earn a classroom of kids and extra quarter million dollars study, from this past winter. Many policymakers leaped to use that study as an immediate call to use value-added data for teacher de-selection policies. That call was endorsed by one of the authors own media quotes in which he asserted that we should fire sooner than later! (and that assertion was built on an overly bold if not absurd extrapolation of the earnings effect based on the single age at which the earnings effect was largest).

Similar overreaching for immediate policy implications appeared in the author-endorsed media spinning of Roland Fryer’s piece on “no excuses” charter schools in New York City, where despite not even attempting to accurately measure school expenditures, or the cost of “no excuses strategies” Fryer  fueled the media assertion that “no excuses” strategies and NOT money are the answer to improving urban school performance (partly in language embedded in the working paper itself).

If we are willing to accept these types of bold immediate policy recommendations, then we might actually be willing to accept the school I’ve laid out above as a reasonable proposition. My research based academy above might actually produce some marginally greater value-added estimates on student achievement data than it would for the same group of kids if I didn’t strategically carb load on testing days, change the kids names (to alter teachers’ expectations of them) and threaten their teachers with complete loss of salary.

And it might even be a really efficient approach to value-added gains if my (completely ridiculous) assumption holds that I can find a pool of average 4^th year teachers willing to enter such a toxic environment for a single year, every year at an average 4^th year salary. Yeah… that’s one $#!+load of assumptions (worthy of a few pages of appropriately formatted footnotes!).

But I’m pretty sure it would be a really sucky place to work as a teacher or to attend as a student. And call me a sappy, post-empiricist, sucker, but that matters too!