NJ Spotlight today posted a story on upcoming Task Force deliberations and public hearings over whether the state should continue to target funding in its school finance formula to local districts on the basis of counts of children qualifying for free or reduced priced lunch.  That is, kids from families who fall below the 185% income threshold for poverty.

The basic assumption behind targeting additional resources to higher poverty schools and districts is that high need districts can leverage the additional resources to implement strategies that help to improve various outcomes for children at risk. I have discussed this issue at length in this related post.  New Jersey has done this better than most states over time. (evidence on outcomes here)

The idea is to find the indicator or measure that seems to best capture the likelihood that children will struggle in school – that they will enter kindergarten less prepared and have access to fewer out of school resources during their time in school (including limited summer learning opportunities). A variety of socioeconomic indicators might be considered. But often, the information that happens to be most available is counts of kids who are from low income families, as identified through the National School Lunch Program income criteria.  And, as a measure of convenience, it tends to work quite well. I compare this measure below with Census poverty measures, based on children in families living in a certain area (within school district boundaries) who fall below the much lower income threshold of 100%, which has some advantages but also some major shortcomings.

Of course, in the political context, this is really all about finding ways to deliver more aid to districts whose representatives/political leaders wield the most power in the political debate. That’s just the nature of the beast – the politics of school finance. Sometimes it goes well for the kids who need it most… other times, not so much.  I watched this play out in Kansas a few years back, and have seen similar conversations occur across other states.

Typically, the whole thing plays out according to the following politically motivated steps:

1. Manufacture some scandalous but largely irrelevant, anecdotal manifesto about how local district officials are egregiously mislabeling children as low income in order to hoard and misappropriate obscene sums of state aid.
2. Manufacture other claims that poverty really doesn’t matter anyway and certainly these poverty measures have little or nothing to do with determining whether children are likely to do well in schools.
3. Assign a task force composed mainly of lay people with little or no expertise in education policy, finance or specifically the measurement of poverty, to swallow whole the manufactured evidence and generate politically convenient policy recommendations.

As I mentioned, Kansas went through this process while I lived there – establishing an “At Risk Council,” and now New Jersey is headed down a similar road. In Kansas, the political strategy of using the Task Force to reduce poverty based funding and drive more to the suburbs was thwarted by the assignment of a knowledgeable individual to head the task force – or At Risk Council – former Commissioner Andy Tompkins. In the end, Tomkins and the Kansas At Risk Council concluded:

The Council continues to believe that the best state proxy for identifying at-risk students is poverty, whether that be measured by free or free and reduced price lunches.

Darn them. Blasphemy! Amazingly, Andy Tompkins was not exiled from Kansas for his leadership in this matter, and he remains one of the kindest, most thoughtful individuals with whom I’ve ever interacted on state education policy issues!

Report here: LEG At-Risk Council Report SFFF

Of course, Kansas legislators still found additional clever ways to shift money away from higher need and toward lower need districts.

In any case, even though no-one asked me… nor do I really want to be asked to participate in such a charade, here are the questions and considerations that should guide the choice of measures for determining state aid distribution.

Two Key Questions:

First, the questions… and the data on New Jersey schools and districts.

Is the Poverty Measure Correlated with Other Poverty Measures?

It is indeed desirable to find some measure on which to base funding allocations that can’t be gamed, or manipulated by those who stand to receive the additional funding. But that’s not always feasible (or cost effective). And, even if a count method does involve local district officials gathering data, it can still be checked/audited (in a  more thorough and responsible way than checking a smattering of individual families forms for those who fall closest to the income threshold, necessarily ignoring those who fall just the other side of the threshold but didn’t file).

One reasonable way to evaluate district collected data on children qualifying for free or reduced lunch is to evaluate the relationship between the free/reduced lunch concentrations and census poverty estimates based on resident populations. Here are three versions of that comparison:

Figure 1. Relationship between Census Poverty 2010 and District Free/Reduced Lunch 2011

In this first figure we see that Census poverty rates tend to range from 0 to about 45% and free/reduced rates – children in families under a much higher income threshold, up to about 100%. In fact, as I’ve noticed in many analyses, the free/reduced lunch data tend to get messy above 80%, suggesting that this is the range within which local administrators may be maxing out their ability to get parents to comply & file paperwork. Here, we see that even though poverty rates keep climbing, free/reduced rates seem to level off. Arguably, if anything is going on here, it’s that very high poverty districts like Camden and Trenton – which fall “below the curve” are under-reporting their free/reduced rates – with some possibility of marginal over-reporting in Elizabeth.

Overall, however, census poverty explains nearly 90% of the variation in free/reduced rates. In other words, free/reduced lunch makes a pretty darn good proxy.

In this second figure, I’ve tried to better tease out the districts that may be under or over reporting by cleaning up that non-linear relationship and expressing both measures in their natural logarithm form. Here, we see that the relationship remains very strong and still slightly curved. If there were districts substantially over-reporting free/reduced lunch, they would appear to pop above the outer/upper edge of the curve. There’s not much of that going on. On the other hand, there are a number of districts that are relatively low in poverty but report disproportionately low free/reduced lunch rates – that is, under-reporting.

Figure 2. Logged Relationship (natural log) between Census Poverty and Free/Reduced Lunch

In general, these figures show that free/reduced lunch rates are a pretty darn good proxy for district poverty rates. And at least this analysis here doesn’t indicate substantial, systematic (beyond predicted, based on resident child poverty rates) mis-classification.

Is the Poverty Measure Correlated with Student Outcomes?

The “big question” is which version of the measure better captures differences in student outcomes – or predicts educational disadvantage.  This is straightforward enough to check as well. The first figure hear shows the relationship between free/reduced lunch rates and proficiency rates on state assessments in 2011.

Now, I know, we’ve been told that this relationship doesn’t really exist. There are lots of schools that flat out buck this trend, right? So much so that it’s not even a trend, right? In fact, we’ve even been fed a totally absurd graph which purports to validate that free/reduced lunch really doesn’t relate to performance.  Oh wait… and we’ve been fed even more ridiculous graphs to reinforce this point!

Setting aside all of that stuff, Figure 3 shows that % free/reduced lunch alone explains about 81% of the variation in proficiency rates across districts.  So, it’s a pretty reasonable proxy of educational disadvantage.

Figure 3. Free/Reduced Lunch & Proficiency in 2011

Now, I do have some concerns about the extent to which this relationship erodes at and approaching free/reduced rates above 80%. Is it really that Camden and Trenton perform that poorly compared to Union and Elizabeth despite serving even less poor populations? Or might the story be more complex than this. Figure 4 which shows the relationship between Census Poverty and proficiency sheds some additional light on this issue.

Figure 4. Census Poverty and Proficiency

Figure 4 suggests that Camden and Trenton are actually a) higher poverty than Elizabeth (and Camden higher than Union) and b) perform more or less where they are expected to [somewhat below… as opposed to well below]. This is an interesting contrast that adds some support to my speculation above that these very high poverty cities may in fact be understating their poverty rates in their free/reduced lunch data. Indeed, there may be some overstating in Union and Elizabeth, but neither “popped” substantially above the curve in the previous charts.

Census poverty rates, while capturing a unique story of difference between Camden and Trenton vs. Union and Elizabeth do slightly less well at explaining variations in proficiency rates, making the free/reduced count preferable in this regard.

Additional Policy Considerations:

Given all of this, there are a few additional considerations when pondering which measure to actually use in state school finance policy.

More Stringent Count Methods require Larger Weights

First, if we choose to use a more stringent income threshold for poverty, like the census poverty measure, we would need to assign the appropriate weight to drive the appropriate amount of funding to high need districts. Simply changing our method of counting kids in poverty doesn’t change the needs of Camden or Trenton. It merely recasts those needs with an alternative measure. More stringent measures require larger weights, an issue that has been explored empirically.

The applies to the choice of using free lunch (130% income threshold) as opposed to free or reduced lunch. Using free lunch only might permit better differentiation between high poverty districts, but a higher weight would then be required to drive sufficient funds to those districts.

Problems with Residential/Geography Based Measures in New Jersey

Census poverty measures are limited in their usefulness in the current New Jersey policy context, because they are based on location of residence and linked to geographic boundaries of school districts. New Jersey has significant numbers of non-unified, regional secondary school districts for which poverty estimates may be imprecise or inaccurate.

Further expansion of charter schools and inter-district choice programs complicates use of measures based on place of residence. Funding to schools must be sensitive to the demographics of students enrolled in those schools.  It would be entirely inappropriate, for example, to require a sending district like Newark or Camden to pay charter or other district tuition on the basis of their own average resident poverty rate if the charter school or receiving district is not taking a comparable share of children in poverty. This is certainly the case in Newark.

As a result, free or free and reduced price lunch measures remain preferable.

So, that’s my 2 cents (okay, more like a few dollars worth) of advice on this issue.

More on this later, no doubt, when the Task Force releases its final recommendations.

Related paper on poverty measurement.

http://aefpweb.org/sites/default/files/webform/VOL%20I-POVERTY%20REPORT-METHODOLOGY%202011-21-12%20CLEAN.docx