The New York Times’s Nicholas Kristof, in “Bill and Melinda Gates’s Pillow Talk,” writes: “It has been 15 years since Bill and Melinda Gates created what is now the largest foundation in the world. This milestone seemed the right moment to ask them what they have learned from giving away $34 billion, what mistakes they have made, and what they disagree about.” Although the Gates Foundation shouldn’t get all of the credit, according to Kristof’s “conservative back-of-envelope calculations,” more than 33 million children’s lives have been saved since the foundation was established.

Of course, mistakes were made in their philanthropy as they “started out too tech-focused.” Now, says Kristof, some of their most effective measures, like promoting breast-feeding, are “distinctly low-tech.”

The Gates’s acknowledge that the foundation’s investments in education in the United States haven’t “paid off as well.” But, Kristof is hopeful regarding “one giant leap” in policy. The foundation plans to “also invest nationwide in early childhood programs.”

I’m not qualified to assess the effectiveness of international aid projects. But, to understand why the Gates Foundation’s efforts at home are not working, we should read the foundation’s (and Kristof’s) descriptions of their programs and ask about the two watch dogs who didn’t bark. The first clue is found in Kristof’s seemingly innocuous statement that the foundation “didn’t appreciate how hard it was to translate scientific breakthroughs into actual progress in remote villages.”

When it first committed to international philanthropy, the foundation may have not grasped the full complexity of the problems it tackled, but at least it built on a foundation of scientific research. In our equally complicated domestic edu-politics, the foundation ignored a huge body of social science and education history. Instead, it built multi-billion dollar initiatives based on the opinions of non-educators who were true believers in “Big Data.”

The Gates’ education policy was based on the hunches of economists who used a lot of sophisticated tools that were invented by social scientists. But these true believers in Big Data adopted methodologies that were inappropriate for guiding social policy, and dismissive of or ignorant about peer reviewed education and cognitive science research. Neither were they shy about their contempt for education practitioners.‪

In fact, the research that informs the Gates’ policy is shockingly unconcerned about the actual realities within schools. Instead, it focuses on incentives and disincentives that it is assumed will test, sort, reward and punish schools to the top. Given the role the researchers hope such incentives and disincentives will play, one would think that they would consider the history of failed education merit pay experiments. They should have also addressed the cognitive science literature which explains why a cornerstone of their model, performance pay, may be doomed as a method of improving schools.

When surveying Gates-funded web sites, the first thing that jumps out at this former academic historian is the lack of falsifiable hypotheses. This is not a minor wonkish concern. It is a result of the foundation’s heavy reliance on regression analyses, as well as their hurried insistence that they did not have the time for a dialogue with educators. The assumption was that they could run controls on their data and that would be a substitute for articulating a hypothesis that would then be subjected to peer-reviewed evidence and analysis. Perhaps, some day, that methodology may become appropriate for studying something as complicated as teaching and learning, across all types of schools, serving all types of students, across a diverse nation. But, also missing from the Big Data analyses are other indispensable academic conventions.

When traditional social scientists publish a social policy study, they used to start with an objective review of the literature, and a respectful summary of the arguments of the various participants in the preceding scholarly debate. They articulated the operative mechanism that would drive the policy that they were evaluating. In doing so, they sought to ground their data in the real world that they sought to influence. The social science method may not be perfect, but it ensured a clash of ideas, and set the ground rules for honest debates.

The even bigger issue that should jump out to all stakeholders is the chronology of the Gates’ efforts. The foundation disproportionately staffed the incoming Obama administration’s education team. They led a dramatically successful political campaign which quickly resulted in the corporate school reform agenda becoming law in over forty states. They did not study the merits of their opinions about school improvement until after they became national education policy. And, that is part of the reason why the rushed Gates policies have produced so little good and so much strife.

Moreover, even when the foundation began its after-the-fact research into the merits of its policies, Gates used a methodology that would be fine, for instance, in theoretical economic studies but that was hopelessly inappropriate to guide education policy. The best example was the $40 million Measures of Effective Teaching (MET) project.  The MET finally concluded that effective teaching can be measured. The real question, however, should have been will it be measured in a manner that improves or damages classroom instruction?

The MET found that value-added estimates of teachers’ effectiveness, as a part of an incredibly expensive multiple measures evaluation system, can be accurate enough to inform macro economic debates or to even be a starting point for eventually understanding the nature of good teaching. It did not find evidence that these statistical models could be made valid and reliable enough for holding individuals accountable. It found no reason to question the conventional wisdom that these quantitative measures would encourage rote, basic skills instruction and promote the education malpractice of drill and kill.

The worst part of the Gates’ agenda is the perverse incentive to increase primitive, soul-killing, worksheet-driven instruction. Even if the foundation was unaware of why this unintended outcome would be most likely to infect high-challenge schools, it should have understood the impossibility of implementing value-added evaluations as Common Core testing was introduced. Even if the foundation didn’t understand why value-added models would not be able to control for the tougher challenge of raising college readiness test scores in schools that serve low-skilled students, surely it was not completely blind to the reasons why the mixing of old bubble-in scores with Common Core test scores would likely be indefensible.

The MET used a sample of students who were far more affluent than those who attend urban schools, and it barely addressed the issue of evaluating high school teachers. It was incapable of controlling for peer effects, which is not a minor point when studying tweens and teens. Its methodology was not adequate for addressing sorting, which is not a minor point in our highly segregated schools. Its findings gave no reason to believe that its value-added component could be made valid for schools with high proportions of English Language Learners (ELLs), special education students, poor students, or high-performing students who were already posting high scores. Had foundation scholars been required to put their assumptions into words as a part of formulating a traditional hypothesis, would they have developed a better appreciation of their model’s flaws in terms of improving actual schools?

A prime purpose of the teacher quality effort was improving instruction in low-performing schools, but that was perhaps the key area where the refusal to articulate hypotheses led to an illogical policy. The value-added portion of the Gates model is inherently biased against inner city teachers. How can an accountability system that is unfair to teachers in high-challenge schools become an incentive for teaching in those schools? Why would teachers commit to schools where they have an unknown but significant chance of having their career damaged or destroyed, each year, due to mathematical error?

Even if billions of extra dollars fell from the sky to pay for the Gates’ approach to teacher quality, there was no reason to believe that there are enough hours in the day for overstretched schools to properly implement it, while also complying with the Obama administration’s increased mandates for testing and Common Core or, conversely, science-based policies like early education.

And, strangest of all, the Gates model was imposed on all teachers – not just the 25% or less who teach tested subjects. This could only be addressed by the worst possible scenario – expanding high stakes testing – or the most surrealistic approach – making up metrics for evaluating art, music, physical education, and other untested subjects. Often the result was the weirdest option in that bizarre method. Teachers were evaluated with the test scores of students who they did not teach and often never met!

That brings us to the second watch dog that didn’t bark and the other difference between the Gates’ international efforts and school reform at home. The Third World’s medical systems were not all well-oiled and efficient, but we don’t hear about philanthropists bragging about starting a war against health care providers who don’t measure up. On the contrary, the foundation and other international health activists focused on the problem – improving the welfare of poor people – not the mass firing of service providers.  So, why did the Gates’s international efforts concentrate on problem-solving as its domestic education reforms dovetailed neatly with corporate school reformers’ campaign to defeat teachers’ unions and the professional autonomy of educators?

Surely, the foundation realized that it was jumping into an education civil war, and that it was siding with newcomers to public schools who sought to blow up the “status quo;” i.e. local school governance, the influence of education schools, the power of teachers’ unions, and the professional cultures of practitioners. They advanced their agenda by demonizing their opponents and others who questioned their demand for hurried “transformational change.” The foundation, however, seemed completely uncurious about the educators’ side of the argument or our professional judgments regarding solutions.

Whether it was promoting small schools, charter schools, value-added teacher evaluations, or Common Core, the foundation seemed oblivious to the interconnected nature of the problems that its policies were supposedly seeking to address, or the contradictory nature of the solutions it favored. In a less toxic environment, the Gates’ small school effort and even its support of charter schools would not be a serious threat to public education. Of course, these choice schools created neighborhood schools with more intense concentrations of students from generational poverty who have endured extreme trauma. Obviously, that makes it much harder for these even more challenging schools to meet value-added test score targets. However, we are also in a reform era driven in large part by closures and turnarounds of low-performing schools, where charter management organizations “cream” the easier-to-educate students in order to win the battle for higher test scores. The combination of these policies constitutes an existential threat to the careers of teachers whose only crime was committing to teach in the toughest schools.

I must close, however, with the way that Gates’ hunches placed high-risk students at even greater risk. I wonder how many of the foundation’s administrators understood the brutality of using a Common Core college readiness test to retain 3rd graders. I can’t understand why they have been so cavalier about the denial of high school diplomas to kids who couldn’t pass college readiness graduation exams. Some reformers now retort that the numbers of children who have been thus punished is low. But, I hope they understand why that is the case.

‪To date, the damage already done by the Gates gambles has been countered by the rebellion of teachers and a grassroots uprising by parents have challenged the test and punish regime. Had we not done so, k-12 schools might have already endured the type of agony that was inflicted on GED test-takers. Since the GED high school equivalency exam became a Common Core test, the number of persons passing those tests has dropped by 90%.

So, Bill and Melinda Gates may believe they can gradually shift gears, start to focus on more humane science-driven policies like early education, and wait for the anger of educators towards them to recede. To date, however, we have mostly forced policy-makers to kick the can down the road, to delay the punitive component of the Gates model of school improvement. We must drive a stake through the heart of the test and punish mentality. And, in many or most states, stakeholders understand we can’t weaken. It will take more than some pillow talk before parents and teachers trust the Gates Foundation to learn from its mistakes.

What do you think? Will the Gates Foundation learn? Could it even apologize? Or, will they shift gears and hope that the failures of the last 15 years will be forgotten?