Harvard professor Thomas Kane recently testified on behalf of the ‘families’ in the Vergera vs. California case.  The plaintiffs in this case seek for the teacher tenure process and the LIFO layoff priorities to be changed in order to ensure that students in California have more of a chance to be taught by an ‘effective’ teacher.  Anytime Kane is involved, we are sure to see him claim that his value-added calculations truly measure teacher quality.

In an LA Times article titled ‘White students get better teachers in L.A., researcher testifies’ they report that when Kane testified in the Vergera trial the other day he explained how his research proved that “Black and Latino students are more likely to get ineffective teachers in Los Angeles schools than white and Asian students,” the article continues:

 In the study, professor Thomas J. Kane concluded that the worst teachers—in the bottom 5%–taught 3.2% of white students and 5.4% of Latino students. If ineffective teachers were evenly distributed, you’d expect that 5% of each group of students would have these low-rated instructors.

The first problem with this claim from this non-peer reviewed study is the implication that teachers who are rated ‘effective’ through a value-added calculation in a school with a wealthier population will, necessarily, still get that same ‘effective’ rating if they were to transfer to a poorer school.  Also implied is that if some of those ‘ineffective’ teachers at the poorer schools were to transfer to the wealthier schools, they would still be ‘ineffective.’

Professor Douglas Staiger from Dartmouth has co-authored many papers with Kane.  Two years ago when analyzing the value-added of teachers in New York City they found that ‘effective’ and ‘ineffective’ teachers were equally distributed among the richer and the poorer schools.  GothamSchools reported this in an article called ‘Why it’s no surprise high- and low-rated teachers are all around’  As this contradicts what Kane has observed in Los Angeles, it is interesting to see this section about the limitations of value-added to compare teachers of different populations, including an admission of this limitation from Staiger:

So while the ratings were explicitly designed to compare teachers who work with similar students, they cannot compare teachers who don’t. “This is just a difficult question that we still don’t know how to answer — this question of how to compare teachers who are in very different kinds of schools,” said Douglas Staiger, a Dartmouth College economist.

But even if you still believe that value-added can be used to compare teachers of different populations, there is a mathematical deception that Kane used in presenting his numbers.  He says that if the bottom 5% of ‘ineffective’ teachers were perfectly equally distributed, you’d see the white kids having 5% ineffective teachers and the Latino kids having 5% ineffective teachers.  This is true.  Instead what we see is white kids getting 3.2% and Latino kids getting 5.4%.  So it can be argued, as Kane does in his slide show, that:

Aside from the slide, the article does not mention that black students also get a lower than average share (4.6% — it doesn’t say this directly, but 43% more than 3.2% is 4.6%) of ‘ineffective’ teachers since I guess this would weaken his testimony.

Kane is accurate when he states that 5.4% is 68 percent more than 3.2%.  But notice that 5.4% is very close to the ‘fair’ amount of 5%.  While 3.2% seems to be an unfairly low share.  How can this be?  Well this can be easily explained by looking at the population distribution of Los Angeles schools.  72% of their 700,000 students are Latino while only 10% are white and 10% are black.  There are 45,000 teachers in Los Angeles so about 33,000 teach Latino students while 4,600 teach white students.  Because there are seven times the number of teachers teaching Latino students, a small change in the number of ‘ineffective’ teachers of white kids would change their percent much more than a change in the number of ‘ineffective’ teachers of Latino kids.  Specifically, of the 33,000 teachers of Latino kids, 5.4% or 1776 teachers were rated ‘ineffective.’  If this number were to change by just 100, the number would drop to .4% to 5%.  For white kids, 3.2% of their 4,600 teachers or 147 teachers were currently deemed ‘ineffective.’  But if those 100 ‘ineffective’ teachers were to now transfer and teach the white kids, the percent would jump up by 1.8% to 5% (assuming, that is, that they were still ‘ineffective’ which is actually not likely anyway).  So in a system of 45,000 teachers, 100 teachers, or two-tenths of one percent of the teachers there comprises the entire ‘imbalance’ where Latino students are burdened with their unfair share of ‘ineffective’ teachers.

I don’t know if the defense attorneys were mathematically savvy enough to challenge this misleading use of statistics.  My hope is that there is enough ‘reasonable doubt’ in the accuracy of value-added that the case will be lost for them on that point alone.  Still, it is interesting to see the way Kane presents his numbers in a way which he must know is skewed.