2005

These notes have two purposes: to disabuse naïve conceptions of the role of research in policy development (if you happen to have any, and perhaps you don’t); to illustrate and conceptualize the actual relationship of social science and educational research to policy.

Since we are working in a graduate program that has training for a career of scholarship and research as its goal, and since it is education policy that is the subject of this scholarly attention, it should not be unexpected that at some point we would ask how these two things come together. There is much more to policy making than doing or finding research that tells one how to make policy—far more. In fact, basing policy on research results—or appearing to do so—is a relatively recent phenomenon. Policies are formulated and adopted for many more reasons than what educational and social science research give. Policies may arise from the mere continuation of traditional ways of doing things; they may come about to satisfy a group of constituents to whom one owes one’s elected or appointed position. They may grow out of the mind of an authority in a way that is completely incomprehensible, even to the leader. But increasingly, policies are being viewed by opinion makers, journalists, intellectuals of various sorts, and even a broader public as not legitimate or not deserving of respect or lacking authority unless they are linked somehow to science. In what follows immediately, I have written down a few ideas on this subject. They may orient your thinking in a way that makes the articles you will read more meaningful. It is also my hope that they will help us understand what went on in the exercise last week when we carried out a simulation of how a policy is influenced by research.

In brief, I believe that research is far too limited in its scope of application to dictate policy formulation. Policy options arise in large part from the self interests of particular groups. No matter how broad and solid the research base, opponents of a policy recommendation that seems to arise from a particular body of research evidence will find ground on which to stand where the evidence is shaky or non-existent, and from there they will oppose the recommendation. If “whole language” instruction appears from all available research to result in greater interest in reading and more elective reading in adulthood, opponents will argue that children so taught will be disadvantaged in learning to read a second language or that no solid measures of comprehension were taken thirty years later in adulthood.

So what is the relationship of research to policy? The answer entails distinctions between such things as codified policy (laws, rules & regulations, by-laws and the like) and policy-in-practice (regularities in the behavior or people and organizations that rise to a particular level of importance—usually because they entail some conflict of interests or values--where we care to talk about them as “policy”).

Policy-in-practice is often shaped by research in ways that are characterized by the following:

• Long lag times between the “scientific discovery” and its effect on practice;
• Transmission of “scientific truths” through popular media, folk knowledge, and personal contact;
• Largely unconscious or unacknowledged acceptance of such truths in the forms of common sense and widely shared metaphors.

Thus do we move from views of children that prevailed in the 19th century—tabula rasa, sinful, responding to corporal punishment, having “mental muscles” needing exercise—to views that prevail in the 21st century—learning through rewards either by operant (Skinnerian) principles or by need reduction, having fragile “self-concepts” that must be nurtured to produce high “self-esteem,” etc.—thanks to Thorndike, Freud, Skinner and the rest. (It may be worth remarking upon at this point that research powerful enough to change prevailing views is rare, resembles what Kuhn has called a “paradigm shift”—although Kuhn quite clearly observed that the social sciences are in a “pre-paradigm state” so that there could be no paradigm shifts in the sense he spoke of them—and is usually rejected as wrong when first presented precisely because it challenges prevailing views. I hasten to add, however, that the vast majority of ideas that challenge the prevailing view are in fact wrong.)

With respect to codified policy, research serves functions of legitimating choices, which it can do on account of its respected place in modern affairs. Research is used rhetorically in policy debates. Not to advance research in support of one’s position is often tantamount to conceding the debate—as if the prosecution put its expert on the stand and the defense had no expert. In rare instances so recondite and far from the public’s ability to understand and in instances where a small group of individuals exercises strict control (“contexts of command” as Cronbach and his associates spoke of them) or in places where the stakes are so low that almost no one cares about them, research may actually determine policy. But even these contexts are surprisingly and increasingly rare. (Research evidence is marshaled against such seemingly “scientific” practices as vaccinating against flu viruses, preventing forest fires, the use of antibiotics, tonsillectomies, and cutting fat out of your diet.)

For the most part, research in education functions in a “context of accommodation” where interests conflict and the stakes are relatively high. There, research virtually never determines policy. Instead, it is used in the adversarial political process to advance one’s cause. Ultimately, the policy will be determined through democratic procedures (direct or representative) because no other way of resolving the policy conflicts works as well. Votes are taken as a way of forcing action and tying off inquiry, which never ends. (As an observer once remarked concerning trials, they end because people get tired of talking; if they were all conducted in writing—as research is—they would never end.)

Research that is advanced in support of codified policy formation is at the other end of the scale from the kind of scientific discoveries earlier referred to that revolutionize prevailing views. They resemble what Kuhn called “normal science”—small investigations that function entirely within the boundaries of well-established knowledge and serve to reinforce prevailing views. Such work is properly regarded less as “scientific” inquiry than as other things: rhetoric, testimony for the importance of a set of ideas or concerns, existence demonstrations (“This is possible.”). [Having written this paragraph in the first draft, now, in the second draft, I have no idea what I was driving at.]

Research does not determine policy in the areas in which we are interested—human services, let’s call them—in large part because such research is open ended, i.e., its concerns are virtually unlimited. Such research lacks a “paradigm” in the strict Kuhnian sense: not in the sense in which the word has come to mean virtually everything and nothing in popular pseudo-intellectual speech, but “paradigm” in the sense that Kuhn used it, meaning having an agreed upon set of concepts, problems, measures and methods. On one of the rare occasions when Kuhn was asked whether educational research had a paradigm, or any recent “paradigm shifts,” he seemed barely to understand the question. In writing (Structure of Scientific Revolutions), he opined that even psychology was in a “pre-paradigm” state. Without boundaries, a body of research supporting policy A can always be said to be missing elements X, Y and Z, that just happen at the moment to be of critical importance to those who dislike the implications of whatever research has been advanced in support of policy A.

This lack of a “paradigm” for social scientific and educational research not only makes them suffer from an inability to limit the number of considerations that can be claimed to bear on any one problem, it also means that there are no guidelines on what problems the research will address. The questions that research addresses do not come from (are not suggested by) theories or conceptual frameworks themselves, but rather reflect the interests of the persons who choose them. For example, administrators in one school district choose to study “the culture of absenteeism” among teachers; in so doing they ignore the possibility that sabbaticals for teachers might be a worthy and productive topic for research. The politics of this situation are almost too obvious to mention.

What are some functions of research in the process of policy formation?

a) Researchers give testimony in the legal sense in defending a particular position in adversarial proceedings, wherever these occur. (There even exists an online journal named Scientific Testimony, http://www.scientific.org/.) Both sides of a policy debate will parade their experts, giving conflicting testimony based on their research. But not to appear and testify is to give up the game to the opposition.

b) Researchers educate (or at least, influence) decision makers by giving them concepts and ways of thinking that are uncommon outside the circles of social scientists who invent and elaborate them. An example: in the early days of the Elementary and Secondary Education Act of 1965 (the first significant program of federal aid to education), the Congressional hearings for reauthorization of the law were organized around the appearance of various special interest groups: the NEA, the AFT, the vocational education lobby, and so forth. Paul Hill, now a education policy professor at the University of Washington, was a highly placed researcher/scholar in the National Institute of Education in the early 1970s. His primary responsibility was for evaluation of Title I of ESEA, the Compensatory Education portion of the law. Through negotiating and prodding and arguing, he succeeded in having the Congressional hearings for reauthorization organized around a set of topics related to compensatory education that reflected the research and evaluation community’s view of what was important: class size, early childhood developmental concerns, teacher training, and the like. No one can point to a study or a piece of research that affected Congress’s decisions about compensatory education; but for a time, the decision makers talked and thought about the problems in ways similar to have the researchers thought about them.

c) Researchers give testimony (in the “religious” sense of a public acknowledgment or witnessing) to the importance of various ideas or concepts simply by merit of involving them in their investigations. (“Maybe this is important, or else why would all those pinheads be talking about it?”)