A decade ago when a surge of interest in placing computers in classrooms was joined to available public and private money, U.S. schools went on a rampage of buying and distributing devices to students and teachers. At that time I wrote a post about a puzzling fact that was overlooked  or simply ignored:  With increased access to new technologies, there is little reliable and valid evidence showing that these technology investments have yielded gains in student achievement.

That was then and this is now. Amid a pandemic-induced shift of face-to-face lessons to online instruction, the question is irrelevant. The need is that every single public school child must have a device to use for remote instruction.

While there was certainly a digital divide a decade ago even amidst the splurge spending for devices, school closings due to the pandemic have pushed districts to triple-time distribution of devices and Internet connections to families lacking either or both.  Yet, the lack of solid evidence that student computer use–especially now when lessons are online–is strongly associated with increased test scores and other academic outcomes remains puzzling. So I updated that post and publish it now.

After decades of school and classroom use of new technologies, some facts have emerged that puzzle me.

Here’s one.

Since the early 1980s, the federal government, states, and districts—not to mention philanthropists—have invested billions of dollars in wiring schools, buying and deploying machines, and preparing teachers and students to use high tech devices. Nearly all teachers now have access to one or more computers at school. As for students, the number of students per computer across the U.S. has gone from 125 per computer in 1983 to 4 per computer in 2006. Teacher and student access to computers has increased even more in the past decade with thousands of schools issuing computers to each and every student and teacher.

With increased access to new technologies, there is little reliable and valid evidence showing that these technology investments have yielded gains in student achievement.

Why?

One answer is simply that access to machines does not necessarily lead to teachers regularly using high-tech devices in daily lessons. Consider that after nearly 30 years of access to computers in the U.S., based on national surveys and research studies of schools, over 60 percent of teachers are regular users, that is, using computers for instruction one or more times a week (see here). These teachers use interactive white boards, laptops, and hand-held devices to have students do Internet searches, turn in typed rather than hand-written homework, take notes on lectures, watch videos, and other familiar classroom activities. A small sub-set of these teachers, however, do use electronic devices weekly, even daily in far more creative and imaginative ways inside and outside classrooms with their students. 

But the majority of teachers, most of whom–paradoxically–use their home computers a few hours each night, are either occasional or non-users in integrating available machines into their daily lessons.

So one explanation for the first puzzling fact is the flawed assumption that deploying computers to teachers and students will lead to teachers regularly using high-tech devices for instruction. Note that without regular use by teachers, establishing either a correlational or causal link between computers and, say, student test scores, is impossible.

Another explanation for the puzzle of so little linkage between computers and student achievement examines how researchers go about studying the connections between technology and student outcomes.

Many researchers fail to consider that the common designs and methodologies they use to determine linkages between classroom technology use and student achievement cannot capture the inherent complexity and unpredictability of teaching and learning. So researchers use shortcuts to get around that complexity and unpredictability.

I need to unpack the previous sentence. Consider that teaching students involves many factors relating to who the teacher is, what content and skills are taught, and what activities and tasks occur while teaching. Also consider student factors: who they are, what experiences, motivations and interests they bring to the classroom, and what they do during lessons. Then consider the school itself, its organization, culture, and its neighborhood. Finally consider the district, its resources, leadership, and culture of learning or non-learning that it cultivates. All of these interacting factors, sometimes unpredictably, affect classroom teaching and learning.

Yet look at the majority of research designs and methods used to determine the effects of teachers using computers with student. Most common are surveys of teachers and students who report their perceptions of classroom use supplemented by researchers’ descriptions of practices, and interviews with teachers and students. Some researchers set up comparison groups of classes that use computers to study a topic with classes not using computers studying the same topic. Then the classes using and not using computers are pre- and post-tested.

Both research designs have serious defects. Short of establishing an experimental and control design with students and teachers randomly assigned to each group, it is nearly impossible to establish a causal linkage between the use of high-tech devices and student achievement. Such experimental or quasi-experimental designs are uncommon and usually too expensive to mount.

Because surveys, class-comparisons, or studies of small groups of students using computers are less expensive in dollars and labor, thousands of studies have been done since the introduction of desktop computers into schools in the early 1980s. Many show minute gains or “no significant difference” in test scores from student use of computers. The results, however, are correlations—associations between presence of computers and gains in test scores, not evidence that student use of the machines caused a rise in test scores (see here).

Here, then, are two ways to make sense of the puzzling fact over the paltry results in student outcomes of so much investment in high-tech devices and so little return on those dollars.

When one shifts to the evidence of online instruction being associated with academic achievement gains, a similar story unfolds. 

A brief look at the many K-12 and higher education studies that have sought an answer to the question of the effectiveness of online instruction in producing student gains in achievement may help readers.

Scores of studies of outcomes for online instruction have been contested because most have had serious design and methodological flaws. Moreover, many of these studies lumped together full-time virtual schools, hybrids, and online courses, And the results have been underwhelming.  That is where heartburn enters the picture (see here).

Even when researchers over the past few decades have performed meta-analyses of a smaller number of studies that have met higher standards of quality they found that virtual instruction in its various modes, at best, is equivalent to regular face-to-face classroom instruction. At worst, some studies showed less achievement gains than traditional teaching. And keep in mind that these meta-analyses were of studies where online instruction occurred in mostly math, reading, and science courses—not other academic subjects. Nor in areas of great concern such as kindergarten and primary grades, the arts, and social and emotional learning. The overall picture is considerably dimmer than promoters of full- and part-time virtual schooling have promised or leaders had expected. *

So with such little evidentiary support for huge expenditures of school budget funds for hardware, software, maintenance, and professional development, why keep buying the stuff? 

There was one explanation I omitted when I wrote this post in 2010 because it did not register with me then but over the following years has emerged as a powerful reason for more classroom use. 

The rationale for giving devices to every student and teacher has shifted from linkage to student outcomes to the simple fact that all standardized testing will be online.  With the Covid-19 pandemic, insuring that every student had access to a device and the Internet took precedence over teacher’s decisions to use or not use devices for their lessons. Until in-person schooling resumes for most U.S. students, all lessons will continue to be online–for better or worse.

____________________

*Cathy Cavanaugh, et. al. “The Effects of Distance Education on K–12 Student Outcomes: A Meta-Analysis.” 2004 Naperville, IL: Learning Point Associates ; Rosina Smith, et. al.  “A Synthesis of New Research on K-12 Online Learning”.2005, Naperville, IL: Learning Point Associates; Barbara Means, et. al., “Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies,” (U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, 2010).