I was reviewing the past few days of news coverage on NJ teacher evaluations and came across the following quote, which was not-so-amazingly left unchallenged:

Cerf said research shows test scores are “far and away” the best gauge of teacher effectiveness, and to not use test score data would be “very anti-child.”

http://www.nj.com/news/index.ssf/2013/05/state_board_of_education_adjus.html

Here’s a reporters’ guide to follow up questions….

Mr. Cerf… can you show me exactly what research comes to that conclusion? (this should always be the immediate follow up to the ambiguous “research shows” comment)

Exactly how is “far and away” measured in that research?

And what is meant by “best gauge of effectiveness?”

That is, what is the valid measure of effectiveness against which test scores are gauged? (answer… uh…test scores themselves)

So, Mr. Cerf, are you trying to tell me that the Gates MET study proved that test scores are “far and away” the best gauge of teacher effectiveness? (seems to be most common reference point of late)

Can you show me where they said that?

And how did they measure what was the best predictor of effectiveness? In other words… what did they use as the true measure of effectiveness?

So… you’re telling me that the Gates study found – not far and away, mind you – that test score based measures are, well… the best predictor of themselves a year later? Is that right?

That’s what you mean by best gauge? Right? That they are the best predictor of themselves… if we also use other measures to try to predict test scores? Right? Seems a bit circular doesn’t it?

And how well did test-score based measures predict themselves a year later? if we accept as logical that validity test?

Well, that seems like a rather modest relationship from year to year, doesn’t it?

How does that make test scores the best predictor of actual effectiveness if actual effectiveness is broader than test scores themselves?

Okay… moving on… Since we’re leaning on those Gates foundation findings as providing the basis for placing heavy weight on test scores in NJ teacher evaluation… I note here (pointing to NJDOE documents on SGPs) that New Jersey has chosen an approach called Growth Percentiles to measure teacher effectiveness…. Can you show me in the Gates studies where the authors find this approach to be appropriate – or even more specifically “far and away” the best approach for measuring teacher effectiveness?

I don’t see any reference to SGPs or MGPs in the Gates studies… why is that?

Are SGPs and VAMs the same thing? I’ve been told there aresubstantive differences.

Isn’t one of these, SGPs, not even designed to isolate the effect the teacher has on test score gains?

In which case, how can they possibly be “the best gauge of teacher effectiveness?”

Let’s save the “very anti-child” stuff for another day!