On October 15th, 2012 Bill and Melinda Gates visited Eagle Valley High School in the Eagle County school district in Colorado. Eagle County, in which ski mecca Vail is located, is a demographically unusual place with almost exactly 50% white students and 50% Hispanic students. Many of the Hispanic students have parents who work in the various ski lodges and resorts.  As the Gates’s only visit one school a year, this one was surely carefully chosen.  Three months later the school is mentioned by Gates in two very high profile places:  A Wall Street Journal article written by Gates, himself, called My Plan To Fix The World’s Biggest Problems and The Bill Gates 2013 Annual Letter.

Here is a quote from the Wall Street Journal article:

In October, Melinda and I sat among two dozen 12th-graders at Eagle Valley High School near Vail, Colo. Mary Ann Stavney, a language-arts teacher, was leading a lesson on how to write narrative nonfiction pieces. She engaged her students, walking among them and eliciting great participation. We could see why Mary Ann is a master teacher, a distinction given to the school’s best teachers and an important component of a teacher-evaluation system in Eagle County.
Ms. Stavney’s work as a master teacher is informed by a three-year project our foundation funded to better understand how to build an evaluation and feedback system for educators. Drawing input from 3,000 classroom teachers, the project highlighted several measures that schools should use to assess teacher performance, including test data, student surveys and assessments by trained evaluators. Over the course of a school year, each of Eagle County’s 470 teachers is evaluated three times and is observed in class at least nine times by master teachers, their principal and peers called mentor teachers.
The Eagle County evaluations are used to give a teacher not only a score but also specific feedback on areas to improve and ways to build on their strengths. In addition to one-on-one coaching, mentors and masters lead weekly group meetings in which teachers collaborate to spread their skills. Teachers are eligible for annual salary increases and bonuses based on the classroom observations and student achievement.

The program faces challenges from tightening budgets, but Eagle County so far has been able to keep its evaluation and support system intact—likely one reason why student test scores have improved in Eagle County over the past five years.

Gates has given a lot of money to Eagle County school district over the years.  They seem to be one of the first districts to implement merit pay based on standardized test scores, over ten years ago.  In addition to teachers being eligible for bonuses based on their evaluations, which include test score ‘growth’, they claim, on their website, that “Eagle County Schools abandoned the “lock-step” salary schedule in 2002.”  They also offer annual performance bonuses, based, in part, on test scores as a component of their PEAR (Professional Excellence, Accountability and Recognition) evaluation system.  So this seems like an interesting Petri dish for ed reform here.  A school system that has a merit based salary scale and also merit based bonuses based, in a large part, on test scores which, Gates claims, has had increasing test scores for five consecutive years.

The first thing I looked into was in what way this merit based salary scale was implemented.  The ‘lockstep’ salary scale that most districts currently have has new teachers making the least amount of money and then as a teacher gains years in the system she moves up the scale with predictable annual pay increases.  Also teachers often get extra money for having advanced degrees.  The Gates foundation once came out with a report that extra pay for masters degrees was a waste of money, and we often hear from places like The New Teacher Project that we are too often losing our ‘irreplaceable’ young teachers because they are unable to jump to the top of the salary scale after proving themselves.  Arne Duncan once said that he’d like the best teachers to be able to make $160,000 a year.  When I think about the ‘ideal’ merit pay based salary scale I imagine young go-getters getting six figures by their third year while older teachers who start to see declining test gains get pay decreases, in order to pay for the other increases.  A system like this would run the risk of discouraging cooperation among teachers as everyone competes for their share of the pot.

It turns out that Eagle County’s merit based salary schedule is nothing like what I’ve just described.  On their website they describe it as follows “Increases to salary are determined by 0 to 4% for individual performance based on the teacher’s evaluation scores (due to the current budget crisis, the percentage used in payout calculations is currently cut in half), plus a negotiated/inflationary component (varies based on cost of living, funding).”  So what this means is that all the new teachers start with a salary of $35,819 and unlike a ‘lockstep’ system where they would be guaranteed some set increase a year, they can get anywhere from nothing to 2% each year (it had been up to 4% before a budget crisis).  So this means that no third year dynamo is going to be making six figures anytime soon.  Basically it is the ‘lockstep’ system with a small chance that a teacher will get a bit larger of a bump in a year, but also the possibility that the teacher will get no increase.  In New York City teachers get around a 3% increase each year, as a comparison.  This is surely part of the reason that Eagle County has trouble holding onto their teachers as 40% of the teachers there have less than four years of experience.

Ironically, the largest salary increase is a $3,000 award for having advanced degrees, even though The Gates Foundation has said that this practice is wasteful.

There are also merit based annual bonuses.  Teachers can earn bonuses of up to 4% (it was 8% before the budget problem), part based on the ‘growth’ of the students in their school and district (that part was, before 2008, based specifically on that teacher’s students) and part based on their evaluation which includes their student’s test scores.  According to this recent article from the Vail Daily, the average bonus was 2%, which amounts to less than $1,000.  As a teacher, I find it hard to believe that this pay schedule and bonus program would motivate anyone who isn’t already motivated to do a good job teaching.

Aside from salary increases, there are ways to make extra money by being promoted to a ‘mentor teacher’, which is about a $5,000 bonus, or a ‘master teacher’, who gets about a $10,000 bonus.  These mentor and master teachers spend a portion of each day (1/3 of each day for mentors and 2/3 of each day for masters) observing other teachers and giving them feedback.

That teacher, Mary Ann Stavney, that Bill Gates gushed about based on his observation of her class (ironic since the MET study supposedly concluded that observation is not an accurate way to determine teacher quality) makes a grand total, according to this document, of $45,989.45, a bit lower than the average teacher salary of $46,634.

So the merit based pay schedule isn’t much different than the ‘lockstep’ type and the merit based bonuses are trivial.  But they must be doing something right if, as Gates claims, that Eagle County test scores have improved for the past five years.  Well fortunately (for me, at least) Colorado has one of the best data systems in the country for getting statistics like this.  It turns out that for five consecutive years, from 2007 to 2011, Eagle County school did show improvement each year.  It wasn’t a large amount of improvement, though, and had Gates gotten the latest numbers he would have seen that in 2012 they saw drops in both math and writing (science was flat for the past three years, while reading continued to go up), making their six year gains fairly low.

Also, since they have the unusual demographics of 50% white, 50% Hispanic, there is a great opportunity to analyze their ‘test score gap.’  To make these graphs I took the data from the school Gates visited, Eagle Valley High School and averaged them together with the only other high school, Battle Mountain High School.  As these schools have approximately the same number of students and also the same demographics (about 800 students in each school with approximately 50% white, 50% Hispanic), it is mathematically OK to do this.  As can be seen from the graphs, there is a ‘gap’ of at least thirty percentage points.  Also it is worth noting that the Hispanic percentages are about the same as the statewide averages.

Perhaps there still is a miracle district out there proving that these reforms are working, but as far as I can tell Eagle County, Colorado isn’t it.  This episode proves, once again, that even though money can buy a lot of things, money can’t buy me love, happiness, or, apparently, truth.