A few weeks ago I was invited by Matt Barnum to discuss various issues in education reform through a series of letters.  Matt is a TFA alum who is now in law school.  He has written several articles in various newspapers about the complexity of improving education.  Most recently he wrote something about how it is time for TFA to fold.

My first thought was that since we have so much in common, these discussions would not have enough conflict to make them very interesting reading.  Matt says that he is generally on the ‘reform’ side of these discussions, though, so I asked him to ‘start’ the exchange.  It is interesting to be on the receiving end of one of these ‘open letters’ as I’ve initiated so many of them.  I can see why so many people didn’t respond to mine.  By the nature of the process, the recipient is likely to feel and seem like he is on the defensive.

Here is his letter, followed by my response:

3/29/2013

Dear Gary,

It was about two and a half years ago that I started reading your book, ‘Reluctant Disciplinarian’. I had just finished TFA’s institute and was a couple weeks away from my first day teaching English at a middle school in Colorado. I remember thinking your book was a bit cynical and a bit negative.

Fast forward a couple months. I wished I’d taken your book more seriously, and I realized the wisdom of many your views. In retrospect, I had been inculcated, by Institute, in the TFA culture of  ‘high expectations’ and I believed that I was an excellent classroom manager because I could control eight or nine summer-school students when multiple other adults were present. Not as easy in a real classroom, as I soon found out. TFA’s faulty training model is perhaps a discussion for another day.

Anyway, I discovered your blog several months ago, having recently finished up my stint as a TFA corps member, and beginning to get into education policy and then some education writing. I regularly visit your blog for what I consider to be the most clear-eyed, fair-minded traditionalist view. (A note on terminology: I tend to like ‘traditionalist’ rather than ‘anti-reformer’ for obvious reasons.) Unfortunately, though I respect much of what you write, I can’t say I agree with most of it.

So why am I writing you? Because I’m interested dialogue rather than monologues. Because I do fear that each side of the education debate has become an echo chamber. Because I’m hoping to have a meaningful discussion between TFA alumni who have divergent views on education policy.

It might be useful to start with what seems increasingly the biggest fault line between reformers and traditionalists: testing. Diane Ravitch recently said that the “most damaging things happening today stem from high-stakes testing.” I’ve got to admit that I’m baffled by the belief that high-stakes testing is that destructive.

I taught in a school district that was very reform-minded: we were regularly evaluated by principals and half of our evaluation was based on student tests, of which there were four or five high-stakes standardized assessments per year for each core subject. (The district’s former superintendent even wrote a report for the Fordham Institute on the evaluation system used.) I’ve written about some issues with my district’s evaluation system, but the amount of testing was, by and large, a good thing in my view.

What’s surprised me most about the anti-testing backlash is how little research is cited to support traditionalists’ arguments. (As a point of comparison, I’m more sympathetic to the anti-charter movement because there’s a solid research base for opposing charters. Though I don’t – a topic for another day, certainly.) Gary, you’ve long been critical of standardized tests, saying that you don’t ‘put a lot of stake into standardized tests.’ But what is your evidence for this, beyond your intuition and (admittedly extensive) experience?

Take a look at the research. The well-known Chetty study found that increased standardized test scores correlated with better life outcomes. The SAT has been a powerful predictor [pdf] of students’ first-year college GPA, correlating about as well as high-school GPA, which is pretty impressive in my view. There’s also strong evidence that the SAT is a good predictor of a student’s likelihood of graduating college. Yes, it’s a reformer talking point but I happen to think it’s true: standardized tests aren’t perfect metrics, but they are useful ones.

Traditionalists sometimes act as if preparing for a standardized test is a useless activity. Not so. Whether you like it or not, to be successful in many professions students will have to be successful at taking standardized tests. I had to pass a test to become certified as a teacher in Colorado; I’m now in law school and had to take the LSAT and will have to take the bar. Sure, there’s an argument that test prep has gone too far – and I would guess that that’s true in some schools and districts – but there should also be an acknowledgement that the ability to take a test has many real-world uses.

Finally, I think Paul Bruno has made the point well over at the Scholastic Administrator blog: it’s bizarre that many teachers are so opposed to testing when in fact they give high-stakes tests (high-stakes for their students, at least) all the time in their own classrooms. Why are standardized tests so fundamentally different than classroom tests?

I saw Karen Lewis speak at the University of Chicago a couple months ago, and she argued against standardized tests, saying, ‘My students aren’t standardized!’ That got some applause and head nods, but does that really make sense? Presumably what she meant was that all students can’t be measured by the same test – but when she was a teacher did she not give all students the same tests? Don’t you, Gary?

Say you go next door to a colleague, and design a test to give to all ninth grade algebra students, with the intent to assess them all equally, judge your own performances as instructors, and find areas of weakness, both yours and your students’. You’ve just created a standardized test, and yet you’d probably agree that doing so is pedagogically sound. I see standardized tests as scaling this idea.

Yes, we need to be careful with incentives when linking pay to test scores, we need to make sure tests are fair and accurate, we need to avoid narrowing the curriculum, and we need to ensure that teachers have a part in designing the tests (which my old district did to its credit). Of course there’s a lot of work still to be done on these matters, but it is work that can be done. These are issues that can be solved, not ones that warrant trashing high-stakes, standardized tests altogether.

Looking forward to hearing your thoughts,

Matt

My response:

4/6/2013

Dear Matt,

Thanks for taking the time to write to me.

No real educator, certainly not me, is opposed to tests.  As a math teacher I give tests all the time.  Over the years I’ve slowly increased the value of these tests, and now they are nearly 90% of my students’ grades for my eleventh graders (I also teach an elective called math research which is more project based and quizzes only count for 15% of that grade).  I take great pride in my ability to make a great test — one that is not too hard or too easy, one that has just the right amount of ‘skills’ but also challenges my students to prove that they understand the underlying ideas enough that they can apply them to an unfamiliar setting.

During my summer training for TFA in 1991 I remember reading somewhere that in an education utopia there would not be any grades.  Students would study and learn because they were intrinsically motivated and interested in the material.  I remember thinking, back then, that I agreed with this.  But I changed my mind pretty quickly once I became an actual teacher.  If you look through both of my books I wrote about teaching, you will see that tests and students succeeding on these tests are a big component of my philosophy of teaching.  Tests serve a lot of purposes.  For one thing, tests serve as a way to encourage students to study.  Also, tests are a way to reward students who have been diligently doing all their homeworks and punish, indirectly, students who have been cheating by copying homeworks from others.  In my own class it is not uncommon for me to say something like “Last year there was a ten point test question on this topic which a lot of people lost points on for making this mistake, so please be aware of this as you do your homework and study.”  I’d say that I refer to ‘the test’ in one way or another, on average, a few times each period.

At my school I’ve been the chairperson for the eleventh grade math final exam for the past three years.  In this capacity I have to collect questions from a team of teachers and compose a test by sorting out the ‘good’ questions from the ‘bad’ ones.  I teach at Stuyvesant High School which is, by many standards, one of the best schools in the country.  I compose a final which, if I did it right, should produce an average of a little under 90.  If it is way off of that, I have made the test too easy or too hard.  There was a time at my school where the teachers who taught the ‘honors’ eleventh graders didn’t want their students to have to take the ‘regular’ final.  They argued that it was too easy for them and would inflate their grades so they wanted to make their own test.  In the debate over this, which I was on the winning side, it was agreed that the honors students should have to take the regular final.  The main argument that was sold was that it is OK if the honors students ace the final since getting a good grade in honors will help them get into college.  In the back of everyone’s minds, though, was the suspicion that the honors teachers were concerned that their students would not do so well on the ‘regular’ final which would make those teachers look like they were not covering the material they were supposed to.  So, in that sense, I do like ‘standardized’ tests for many purposes.

Like the two people you mentioned, Karen Lewis and Diane Ravitch, I am not opposed to tests — or even standardized tests.  The issue I have is that the scores on these tests are being misused in a way that, in my opinion, will ultimately make education in this country (and even achievement) worse.  This is why I scoff at ‘high performing’ charters who beef up their scores in various ways (and STILL can’t get them to be very good, in most cases).  This is why I oppose value-added being a component of teacher evaluation.

I suppose the biggest misuse of testing data is in teacher evaluation.  I do not believe that ‘bad teachers’ are the problem in this country.  So I really don’t think that fixing the ‘broken’ teacher evaluation system will raise ‘achievement’ in this country.  Instead what will happen is that a lot of money will be wasted (not to mention instructional time) in creating, preparing for, administering, grading, and interpreting the scores.  Despite what we hear from ‘reformers’ from time to time, money does matter and giving more of it to a school, provided they don’t waste it (which is another issue …) is a good thing.  So I don’t like to see billions of dollars wasted on educational alchemy, in this case turning test scores into golden teacher effectiveness metrics which, in theory, would result in teachers learning how to improve and, if that doesn’t work out, how to be fired.

Already we are seeing that even under these new evaluation systems most teachers are being rated ‘effective.’  This is very frustrating to reformers who think that this just means that the new evaluation systems are too lenient.  Never would they think that this might mean that they were wrong about how many bad teachers there were.  This is bad science.  You should conduct an experiment to test a hypotheses and if the experiment disproves the hypothesis you don’t reject the experiment.

I do agree that if there are two teachers at the same school who have generally similar students for a bunch of years and for one of them the test scores on a uniform end of the year test are substantially higher than the other then it is probable that that person is a better teacher.  This is particularly true if that teacher was not ‘teaching to the test’ and, worse, choosing to not expose students to activities that promote critical thinking since that would take time away from preparing for the convergent thinking tests.

But when ‘high stakes’ are attached to these tests, a whole new dynamic opens up that, I think, hurts schools, teachers, and students.  Teachers would scan their class rosters in the beginning of each year wincing when they see that have some students who are disruptive.  Maybe some teachers have an ‘in’ with an administrator who will transfer those students to another teacher.  I’ve read about schools that have a term called ‘bubble kids.’  This means that they have done a ‘triage’ on their students where some will pass the test without little help and some won’t pass the test, even with a lot of help.  Teachers are instructed to focus on the bubble kids who are on the border of passing and failing.  This is an unethical gaming of a system that puts too much stake in ‘percent proficient.’

Value-added based on test scores is too inaccurate for it to really be any part of a teacher’s evaluation.  How can a system which I’ve learned can rate the same teacher in the same year as a highly effective 7th grade teacher yet an ineffective 8th grade teacher.  It makes no sense.  The tests, which I have analyzed on my blog, are not good enough.  For enough money it might be possible to create different assessments for students that might be able to isolate teacher quality.  These tests would be quite expensive to make and to grade, so I don’t see why we should spend all those resources trying to identify the few ‘bad’ teachers that every principal already knows who they are.

You mention the Chetty report that students with teachers who had high valued added had better lives.  But have you seen how easily that report has been refuted?  For one thing, the students with the ‘good’ teachers only made about $250 a year more, on average, and this was after they decided not to count the students who were making the most money — throwing them out as outliers.  Perhaps this is why they didn’t submit it for peer review.

I think the very worst misuse of testing, though, is when it is used to label schools, through a value-added type calculation, as ‘failing’ or ‘high achieving.’  In my research I’ve found that often there is not much of a difference between the two schools, and when there is a difference it is likely because the ‘high achieving’ school (often a charter school) has ‘better’ kids.  Standardized test scores are being used as a weapon to demolish schools, most recently over 50 in Chicago, across the country.  Whatever benefit there might be from high-stakes use of standardized test scores (They motivate the miniscule fraction of teachers who aren’t otherwise motivated to try?  They make some charter operators very rich?) this is far outweighed by the damage they have done.  I’d be very happy if schools stopped getting closed down over standardized test scores and if teacher pay and job security were not linked to it.

My belief is that the misuse of testing is beginning to backfire on the reformers.  With the new systems being rolled out around the country, as part of Race To The Top, it will soon be clear to everyone how far we are from being able to use these test scores any anything more than a very rough idea of where some students are in their knowledge.

Thanks again for writing.  I hope I’ve answered your questions and feel free to write back as often as you’d like.

Gary