The latest National Assessment of Educational Progress (NAEP) scores for the 2019 tests were released on October 30th.  Unlike state tests for which the cutoff scores can be manipulated for political purposes, the NAEP does seem to be somewhat unbiased.  So the NAEP, sometimes called ‘The Nation’s Report Card’ does offer an interesting amount of data that I believe is worthy of analysis.

Often the NAEP results are, intentionally or unintentionally, interpreted to see if it is possible to find some kind of correlation between the education policies a state has enacted and the corresponding NAEP results.  In Obama’s 2014 State Of The Union address, he mentioned that D.C. and Tennessee were improving — as evidenced surely by their NAEP gains from 2011 to 2013 — to show that his Race To The Top recommendations, which were followed closely by those two regions, were working.

So when the 2019 results came out the other day, things looked bad for the reformers.  From 2017 to 2019, the average scale score for 4th grade reading was down 1 point, 8th grade reading was down 3 points, 4th grade math was up 1 point, and 8th grade math was down 1 point.  Though it is not clear to the public whether or not one ‘point’ is a lot or a little, everyone can agree that it is better if the scores go up rather than down.

Only one region had an increased score in 4th grade reading while 17 states decreased.  Only one region increased in 8th grade reading while 31 states decreased.  For 4th grade math, 9 states increased while 3 decreased, and for 8th grade math 3 states increased while 6 states decreased.

Former Secretary of Education, Arne Duncan, wrote an article for The Washington Post called What we can learn from the state of our nation’s education.  He begins by writing:

The latest results from the National Assessment of Educational Progress (NAEP) are prompting some soul-searching about the limited gains over the past decade, but there are outliers worth saluting. More important, we should be analyzing what successful states and school districts are doing differently so that others can learn from them.

He then tells us about some of the ‘bright spots’ which, of course, happen in places like D.C., Tennessee, Louisiana, and Denver — all places that have followed the Race To The Top playbook with charter schools and using value-added to rate teachers.  About Louisiana he says “Louisiana posted nation-leading gains in eighth-grade math” and later credits Chief of Change John White, a former TFAer who has been Louisiana’s State Superintendent of Education since 2012.  He concludes by warning us not to look at the overall lack of improvement as an excuse to rethink the reform agenda he promoted.

The one thing the United States cannot do is use these results as an excuse to go backward to the days when standards and expectations were low. We cannot return to a time when achievement gaps around race and poverty were hidden. We cannot pretend that talent strategies will happen on their own without intentional efforts to recruit, support, retain and hold accountable educators.

I’ve been following Louisiana’s John White for some time and always like to catch him cherry picking data to make it seem like he has helped Louisiana to improve in education.  Debunking this recent falsehood about Louisiana leading the nation in 8th grade NAEP growth was one of the easiest ones to uncover.

Imagine you have a friend who has been on a diet for 6 months.  You ask him how his diet is going and he proudly asserts that he lost 5 pounds in the past month.  But he looks a bit nervous when he is telling you this so you ask the important follow up question:  How much have how lost over the past six months since you started the diet.  He confesses that he hasn’t lost any weight in the six month period and actually gained a pound in that time.  It’s a good thing you knew what sort of follow up question to ask.

So while, yes, Louisiana’s 8th grade math NAEP in 2017 was 267 and their 8th grade math NAEP in 2019 was 272 which was a 5 point gain in that two year period and while that was the highest gain over that two year period for any state, if you go back instead to their scores from 2007, way before their reform effort happened, you will find that in the 12 year period from 2007 to 2019, Louisiana did not lead the nation in 8th grade NAEP gains.  In fact, Louisiana went DOWN from a scale score of 272.39 in 2007 to a scale score of 271.64 in 2019 on that test.  Compared to the rest of the country in that 12 year period.  This means that in that 12 year period, they are 33rd in ‘growth’ (is it even fair to call negative growth ‘growth’?).  The issue was that from 2007 to 2015, Louisiana ranked second to last on ‘growth’ in 8th grade math.  Failing to mention that relevant detail when bragging about your growth from 2017 to 2019 is very sneaky.

This is just one small concrete example of how reformers will cherry pick data to claim that there are bright spots in this NAEP data that show that we need to continue following the lead of people like Arne Duncan.

I think 2007 is a good benchmark year, in general, to look at how much the country ‘grew’ on NAEP.  For 4th grade reading, the scale score decreased by one point between 2007 and 2019.  For 8th grade reading, the scale score stayed the same from 2007 to 2019.  For 4th grade math, the scale score increased by one point between 2007 and 2019, and for 8th grade math, the scale score increased by one point between 2007 and 2019.

There is a lot more to be said about the NAEP results and how to interpret them.  For example, it is hard to compare growth between two states that had different starting points.  Like when someone is 100 pounds overweight it might be easier to lose 10 pounds while if someone is 10 pounds overweight it might be more difficult.  I’ve seen analysis that higher growth (as measured in points) correlates with lower starting scores, which helps explain the so-called ‘Mississippi Miracle’ reformers are now talking about.

Three of the places that Duncan touts for their ‘gains’: D.C., Louisiana, and Mississippi are three of the lowest scoring NAEP regions.  Why should we be looking to them for things to emulate?

Reformers will always look to cherrypick data that they can twist to make it look like they should continue to have the power to influence education policy.  Usually they have to stretch so far to make their claims that they are fairly easy to uncover.