Linked in her article for The Conversation is Sally Riordan’s “Improving teaching quality to compensate for socio-economic disadvantages: A study of research dissemination across secondary schools in England.”

This analysis is another powerful cautionary tale about education reform, notably the “science of reading” (SOR) movement sweeping across the US, mostly unchecked.

As I do a close reading of Riordan’s study, you should also note that the foundational failure of the SOR movement driving new and reformed reading legislation in states is that the main claims of the movement are dramatically oversimplified or misleading. I strongly recommend reviewing how these SOR claims are contradicted by a full examination of the research and science currently available on reading acquisition and teaching: Recommended: Fact-checking the Science of Reading, Rob Tierney and P David Pearson.

This close reading is intended to inform directly how and why SOR-based reading legislation is not only misguided but likely causing harm, notably as Riordan addresses, to the most vulnerable populations of students that education reform is often targeting.

First, here is an overview of Riordan’s study:

Similar to public, political, and educator beliefs in the US, “QFT [quality first teaching] is a commonly held belief amongst school staff” in the UK, Riordan found. In other words, despite evidence that student achievement is overwhelmingly linked to out-of-school factors, teacher quality and instructional practices are often the primary if not exclusive levers of education reform designed to closed so-called achievement gaps due to economic inequities.

This belief, however, comes with many problems:

Riordan’s analysis is incredibly important in terms of how the SOR movement and overly simplistic messaging (see Tierney and Pearson) have been translated into reductive legislation, adopting scripted curriculum, and banning or mandating practices that are not, in fact, supported by science or research.

Riordan identifies bureaucracy and simplistic messaging as the sources of implementation failure:

Nonetheless, “[t]his explicit demand [belief in QFT] is an example of the growing pressure on education practitioners to ensure their practices are supported by evidence (of many kinds),” Riordan explains, adding, “School staff believe that high-quality teaching reduces SED attainment gaps and that their belief is backed by research evidence.”

The research/science-to-instruction dynamic is often characterized by narrow citations or cherry-picking evidence: “Because school leaders cited the same references to research evidence to justify very different policies and practices, I conducted a review of the literature that led to these citations.”

One key problem is that while the evidence base may be narrow and “[a]lthough there is agreement that high-quality teaching is important to tackle SED, principles of QFT are nevertheless being implemented in a myriad of ways across secondary schools in England.”

In the US, many scholars have noted that the SOR movement uses “science” rhetoric but depends on anecdotes for evidence; and, in the UK:

Although many school staff (and particularly school leaders) are aware of the EEF resources and believe that there is evidence supporting principles of QFT, no interviewee described this evidence in any further detail. When asked why QFT works, staff reasoned intuitively. The line of reasoning that can be reconstructed from their replies is independent of the research evidence.

…This intuitive argument, reasoned by school staff, is limited but I do not challenge its validity. The main point here is that this line of reasoning does not reflect the research evidence (which is described in detail below ‘The weakness of the evidence for QFT’). It is not the strength of the evidence base that has convinced school leaders to implement QFT practices. This highlights the importance of the psychological aspects of bringing research evidence to bear on practice. It also raises the possibility that a message was disseminated that was already widely believed. I turn to this bureaucratic concern next.

Improving teaching quality to compensate for socio-economic disadvantages: A study of research dissemination across secondary schools in England

That intuitive urge, again, however, is linked to limited evidence: “Just five studies are being relied upon to disseminate the message that high-quality teaching is the most effective way to reduce SED attainment gaps.”

What may also be driving a misguided reform paradigm is convenience, or a lack of political imagination:

Evidence- or science-based reform, then, tends to be reduced to a “sham” (consider the misleading “miracle” rhetoric around Mississippi, also addressed in Tierney and Pearson):

The unintended consequence is a “misdirection of energy and time of school staff” driven by “pressure to conform to the policies promoted.”

Key to recognize is Riordan identifies that QFT reforms not only fail to close gaps but also cause harm: Some “attempts to improve the quality of teaching are contributing to a large attainment gap,” including: “It is by turning to a more refined measure of SED that we find evidence that the school’s innovations in teaching and learning over the last five years have benefitted its most affluent students most of all.”

Riordan’s conclusion is important and damning:

It has reviewed the wider picture in which school leaders are choosing to implement (or at least justifying the implementation of) particular practices based on a generic message instead of the specific research supporting those practices. The problem here is that the mechanisms operating to connect research with practice are too crude to acknowledge the richness and messiness of social science research. The message, ‘high-quality teaching is the most effective way to support students facing SED’, is too simple to be meaningful. 

Improving teaching quality to compensate for socio-economic disadvantages: A study of research dissemination across secondary schools in England

For the US, education reform broadly and the SOR movement can also be described as grounded in messages that are “too simple to be meaningful” and thus too simple to be effective and even likely to be harmful.