What is the definition of ‘month’?  Nowadays we think of it as the 31, 30, 28, or even 29 day period of which there are twelve of them in a year.  Long ago, a month was defined as the 29.5 day period it takes for the Moon (from which the word ‘month’ is derived) to make a complete revolution around The Earth.  The Hebrew calendar still has months that alternate between 29 and 30 days so the average month is 29.5 days.  In education research, the word ‘month’ also has an unconventional meaning.

I first learned about this when I had read that there was a study showing that TFA secondary math teachers teach 2.6 ‘months’ more than non-TFA secondary math teachers.  The seemed outrageous to me at the time, so I wrote to one of the paper’s authors and detailed my analysis in a post here.  In short: I learned that non-TFA secondary math teachers are only expected to get students to answer about four more questions correct on a standardized test at the end of the year compared to the beginning of the year, while TFA secondary math teachers are expected to get students to answer about five more questions correct at the end of the year.  Basically, the non-TFA teachers teach almost nothing and the TFA teachers teach 26% more than almost nothing.  But if that ‘nothing’ is called ten ‘months’, then the 26% more is 2.6 more ‘months.’  Basically, the questions right on a standardized test to ‘months’ conversion is a very suspect one.

On June 10th, 2014 in Los Angeles, the initial ruling on the trial Vergara vs. California was handed down by Judge Rolf Treu.  In it, he struck down several laws as unconstitutional including LIFO, the current teacher dismissal process, and the 18 month probationary period before tenure for Los Angeles teachers.  Judge Treu was emphatic as he wrote on page 8 of his decision:

Evidence has been elicited in this trial of the specific effect of grossly ineffective teachers on students. The evidence is compelling. Indeed, it shocks the conscience.  Based on a massive study, Dr. Chetty testified that a single year in a classroom with a grossly ineffective teacher costs students $1.4 million in lifetime earnings per classroom.  Based on a 4 year study, Dr. Kane testified that students in LAUSD who are taught by a teacher in the bottom 5% of competence lose 9.54 months of learning in a single year compared to students with average teachers.

Harvard professor Thomas Kane’s testimony was pivotal in this case.  The Kane 9.54 months of learning lost is quite a statistic.  If ‘month’ is defined the conventional way, as an approximately thirty day period, it would mean that if a school has twenty teachers, the worst teacher’s students would learn a year of school less than the average teacher at that school, who, himself probably doesn’t accomplish a year of learning.  Therefore students who get those bottom 5% teachers will know less at the end of the year than they did at the beginning of the year.It reminds me of an episode from The Simpsons where Principal Skinner is fired and replaced with Ned Flanders who lets the school get out of control for being too nice.  In one scene, Milhouse is squirting ketchup on his stomach and says “This is great.  Not only am I not learning.  I’m forgetting stuff I used to know.”

I got a chance to look at some excerpts from the transcripts and checked to see the context of this testimony.  What I learned is that the 9.54 months was what Kane concluded in a study he did about ELA scores in Los Angeles.  The next thing he testified was that with math, the situation was even more exaggerated, with the students of the bottom 5% teachers ‘learning’ 11.73 ‘months’ less than students who have an ‘average’ teacher.  So why would Judge Treu choose to highlight the 9.54 month statistic rather than the 11.73 month statistic?  I think it is because the 11.73 month statistic is so over-the-top, that it hurts Kane’s credibility.  Even the prosecutor was taken aback by this number:

Q      AND SO WHAT IS THE 11.73 THAT IS LISTED ON THE BAR LABELED LAUSD?

A      SO THAT IS PORTRAYING THE ESTIMATED EFFECT OF BEING ASSIGNED TO A FIFTH PERCENTILE TEACHER VERSUS THE AVERAGE TEACHER IN LOS ANGELES.  WHAT THAT SAYS IS BEING ASSIGNED TO A FIFTH PERCENTILE TEACHER MEANS I LOSE 11.73 OR ALMOST A WHOLE YEAR’S WORTH OF LEARNING RELATIVE TO THE STUDENTS ASSIGNED TO THE AVERAGE TEACHER.

Q      HOW IS THAT POSSIBLE, DR. KANE?  HOW COULD A STUDENT LOSE AN ENTIRE YEAR OF LEARNING IN ONE YEAR?

A      BY NOT LEARNING VERY MUCH.

And they moved on to the next question.  Certainly this was an opportune moment for the defense to object, though they didn’t.  Then on cross-examination, the defense never seems to have challenged him on how ‘month’ was defined, which was a key tactical mistake in the defense, one that I hope they correct on appeal and one that other defense lawyers I hope learn from as copycat Vergara trials pop up all over the country.  They should demand that he gives these difference in terms of percentiles and, to put those into more of a context, an approximation for how many questions different that would be on a fifty question multiple choice standardized test.  If it amounts to one or two questions, as I believe it would, that would definitely be less likely to shock other Judge’s consciences.