Beliefs, opinions, and politics matter more in making policy decisions than applying research findings to schools and classrooms. A recent Canadian study (1905-8719-1-PB ) confirmed what amounts to a fact in U.S. policymaking. Candadian researchers looked at provincial policy elites (in U.S., they would be state-level decision-makers) and district officials (school board members and superintendents) and found across Canada what is very evident in U.S. districts as well: politics and beliefs trump use of research in adopting policies aimed at improving practice (see here and here).

As one would expect in academic circles, the language of applying research findings to educational policies has expanded. Canadian researchers Gerald Galway and Bruce Sheppard note that new phrases have entered the vocabulary: “knowledge transfer, evidence-informed policy, data-driven decision-making and knowledge brokering, to name a few. Knowledge mobilization (KM) has been touted as a useful all-encompassing term because it conveys the notion of direction instead of random interaction and it ’embodies the idea that the use of knowledge is a social process, not just an intellectual task’ ” (p. 9). Whatever phrase is used in Canada or the U.S., the pattern of applying research findings to forming, adopting, and implementing policies remains similar on both sides of the border.

Galway and Sheppard studied “senior bureaucrats” in provincial ministries and local board trustees and administrators at two points in time, 2006 and 2012. They used surveys, interviews,and focus groups to generate the data they analyzed.

A typical response from  “senior bureaucrats” in provincial ministries about research and policymaking was (M refers to “ministry” official):

M1: I guess that what I find with university research is that those who are doing it appear to be somewhat removed from the immediacy of it all; it’s not engaged, as it were, and [university researchers are] somewhat reluctant to engage those who are involved in a particular issue on a daily basis. That’s being pretty general; that’s not the case with all university research.

M7: Well, I haven’t had a chance to look at anything [university researchers] are doing so I really don’t know, but I do find that sometimes, just from past experiences, that the research coming out of (names university) is not always; well not the research, but some of the conclusions they draw from their research is not always applicable….

M2: Well, I don’t really know about it, to be truthful. As I was saying earlier that’s one of the things that is most surprising to me; that with the amount of research that is done at the university, how little discussion or impact it’s had on my work as the Minister in this portfolio.

When the researchers turned to trustees–district school board members–survey results and focus group interviews in the 2012 study produced a list of factors that influenced their making of policy.

[W]e presented trustees with 20 potential factors/influences and asked them to indicate on a seven- point scale, the extent to which each has had an effect on specific decisions or recommendations that they have made as trustees…. The principal six factors are as follows:

1. personal or professional beliefs and values,
2. potential to directly influence student outcomes/student learning,
3. advice of district staff and/or colleagues,
4. the school board’s strategic plan 
5. past experience.

Trustees and superintendents rated the following six factors as having the least influence on decision-making (my emphasis):

1. representations of business/private sector,
2. pressure from special interest or lobby groups,
3. a situation or event someone told you about,
4. pressure from government (ministry/department of education),
5. public opinion/avoidance of negative media attention
6. university-based research.

Analysis of the data by province showed very little variance in the factors and evidence that influence school board decision-making across jurisdictions.

The Canadian researchers conclude:

We conceptualize use of low research impact partly in terms of trust vs. risk avoidance. In both policy contexts – provincial departments of education and school boards – we conclude that decision-making is ambiguous and risky…. [I]n uncertain times, decision-makers value knowledge that is familiar and emerges from their own community…. This information – now repackaged in the form of staff advice – becomes privileged and trusted by ministers and senior bureaucrats as authentic knowledge…. readjustment.

Very little found north of the border, then, differs greatly from what U.S. researchers have found in the 50 states (see here, here, and here).

So what?

Keep in mind that none of the above critiques of limited influence of research on policy is restricted to public schooling. Making policy in systems of criminal justice, environmental improvement–think climate change–and health improvement and clinical medicine–think of TV ads for drugs–are subject to similar political factors and personal beliefs rather than what research has found. Calls for more collaboration between university researchers and policymakers have also been heard and ignored for decades. Critics have pointed out many times that the academic culture and its rewards overlap little with the world that decision-makers face every week.

I wish I had a neat answer to the “so what” question and offer a package of do’s and don’ts for those whose who want evidence-based policymaking.  I am on the lookout for such advice but thus far I have come up empty. The fact is that beliefs, opinions, and politics matter more in making policy decisions than applying research findings to schools and classrooms.