The practice of medicine is not an exact science and that No Guarantees or Assurances have been made to me concerning the outcome and/or result of any procedure.

Consent statement from a medical care provider

Anthony Cody, over on Living in Dialog, was right in questioning the motivation of the Gates Foundation which wants to place a camera in every classroom and to use the video tapes improve teaching. Cameras could be a useful tool for teachers, but not the way Gates wants to use them. As Anthony Cody points out, the Gates Foundation is determined to fund a system that bases a large part of teacher evaluation on student test scores. From Anthony’s blog is the quote made by Gates:

Such a system would include test scores, but would also include classroom observation, parent and student surveys, and videos taken in the classroom. The cost today of adding a web camera to a classroom is very low. And so a teacher should be able to tape a segment where they want advice, they see that a discipline problem, or maybe something interesting, perhaps they could have done better, they can post that, and get advice from great teachers. This measurement system is a very important priority.

In this blog post I want to talk about teacher evaluation and the improvement of one’s teaching. I will argue that using digital cameras can be a two-edged sword, offering teachers good and bad consequences. On the one hand are those who want to use video capture of a few lessons to test teacher performance. This could lead to bad consequences for teachers. On the other hand are those who believe that video should be used locally among teachers in the same department or school for critically studying and experimenting with teaching. Collaboration with mentors and among colleagues has been shown to lead to positive consequences and the potential for improvement of teaching.

The analysis of classroom performance using one of the many observation systems tends to reduce teaching to a list of skills or performances that create a mechanistic view of teaching.

As you read ahead, you will find that there is a drive to turn teaching into an exact science, and to even make statements such as,

We found that more effective teachers not only caused students to perform better on state tests, but they also caused students to score higher on other, more cognitively challenging assessments in math and English (Ensuring Fair and Reliable Measures of Effective Teaching 2013, p. 5).

But as I will show, the teacher evaluation systems that are going forward in the nation today are based on questionable measurements and statistics as well as traditional views of what is student learning and how to measure learning. Innovative and creative teaching does not flourish in a system that is based on a discrete set of teaching variables that may or may not contribute to effective teaching.

Camera, Camera on the Wall, Who is the ….

The idea of putting a video camera in every classroom reminds me of a question a sixth grade girl asked me thirty years ago. I had returned from a trip to the Soviet Union, and upon my return was asked to give a little talk to the sixth graders at our neighborhood school. During the discussion with the students, a student asked me if it was true that all the rooms in hotels in the USSR are bugged?

I told her I didn’t know but did tell her this story. I said that while we were in Moscow we had a meeting with the American ambassador to the Soviet Union, Arthur A. Hartman at the US Embassy. The ambassador confirmed that all the rooms in the US Embassy in Moscow were bugged! However, he said it wasn’t all so bad. When he really wanted to send a message to the Soviets, he simply stood in the middle of a room and talked out loud. The microphones were always on so he was sure his message would be heard. The sixth grade student commented that those listening surely must go home each day with headaches!

The surveillance of citizens using CCTV (closed circuit television) has a long history. The first to use the idea were the Germans in WWII at the V-2 rocket site at Peenemünde, and closed circuit TV was commercialized in the US in 1949. (Wikipedia 2010). CCTV is ubiquitous today.

According to published reports, Bill Gates suggests it would only take $5 billion to put a surveillance camera in every American classroom, like the one shown here.

Will putting cameras in every classroom achieve government’s want to watch what we are doing or provide security? When will they watch? What will they look for?

Can we believe that the reason for putting cameras in classrooms is to improve teaching, or is it to check teacher performance and use student high-stakes tests to judge teacher’s as really good or really bad?

Using video in teaching, teacher education, and staff development is not a new idea. In fact, long before Bill Gates proposed that all teachers in the U.S. be videotaped, teachers and university researchers were using video tape technology in teacher education and staff development.

Education researchers and teachers have embraced more cognitive and humanistic paradigms of student learning moving away from behaviorist and positivistic models of teaching. The system that reformers such as Gates, and most U.S. department’s of education is to rely on behaviorist and positivist models linking student achievement of specific teacher performances.

Classroom Video Technology

Here, very briefly, are some of the ways cameras have been used in the classroom. In most of these cases, the technology described here was used in the service of improving teaching and learning. To impose cameras in classrooms to test teachers is not only counterproductive, but the research basis is very weak. Although a great deal of progress has been made in the development of observation systems of classroom instruction, we are hard pressed to use results to judge or rate teaching. And to link specific teaching behaviors to student achievement is questionable along several fronts, e.g. the reliability and validity of achievement tests as a measure of curriculum. Yet, there is an unswerving goal to use video to do just that.

Bill Gates might mean well, but it seems to me he has a limited understanding of what teachers do to motivate their students. His view of teaching and learning is simple. I observed one of his Ted interviews in which he said that measuring what students learned in a course was actually quite easy to decide. All we have to do is give a test at the beginning of the course, and then test the students again when the course ends. Bingo! All you do now is compare pre and post test scores and there you have it–a number that tells us what the students learned. I don’t think so.

Gates fails to consider variables such as the experience of the teacher, the number of students in this class that fear being deported, are on free or reduced lunch, or simply dislike school. There are many out of school factors that teachers have little or no control that may impact learning more than the competence of the teacher. The difference between pre and post test scores is not a valid measure of what students learn in science class. As a research design, it’s the weakest of the many education research protocols.

Yet video can be an effective tool for teachers only if they are in control of how, when, and why video technology is used in their classroom. Video can be used to gather digital images and movies on different pedagogical practices, of students at work, discussions and debates. Video, used in consultation with teacher-research collaboratives, can inquire into classroom instruction and learning. Video of various teaching methods can be assembled and put up on the web for use in staff development and teacher education.

A camera in every classroom?

But to think that we can install a camera and use videos to correlate teacher behavior with student achievement scores is not only unnerving, but there isn’t much support in the research literature to use video samples which are observed by some hired bureaucratic is unfeasible. What observation instrument will be used? Will the same observation or evaluation instrument be used to judge teacher performance without consultation with the teacher. Videos viewed in an observation booth hundreds of miles from the teacher’s school is without context, emotion, feeling, and an intuitive aspect of teaching. Teaching is as much an art as it is science.

Here are a few other ideas and personal experiences with video for us to consider.

Micro-teaching

The use of video first appeared in the 1960s, especially with the development of micro-teaching at Stanford University. Micro-teaching used video taping to record a short lesson (10 – 15 min) by the teacher that focused usually on teaching skills (asking questions, using examples). After the lesson, the video tape was used to analyze the lesson. According to the developers of micro teaching, the students that teachers teach in micro teaching should be “real” students from the K-12 population.

Micro-teaching focused on the skills of teaching with reflective dialog, and so, it was an effective way of initiating novice teachers or interns to teaching using small group formats. Some of the early uses of micro-teaching included pre-service teachers, in-service teachers, Peace Corps participants, counseling, and college teachers. Most schools of education use variants of micro-teaching in their pre and in-service programs.

My first experience with micro-teaching was in the summer of 1969 at an NSF Earth Science Leadership Conference at the University of Maryland directed by Dr. Marjorie Gardner. Each of us in this three-week institute taught a short earth science lesson to a group of six high school students. A post teaching reflective session was held with a small group of co-participants who observed the lesson, These reflective sessions promoted an atmosphere of dialog and discussion of teaching practice. After a re-teach of the lesson, another discussion of the lesson was held to weigh in on the effectiveness of the suggested changes.

I used various hybrids of micro-teaching in science teacher preparation programs at Georgia State University from 1969 – 2003. Although I preferred having students in our science teacher education program (The TEEMS Program) spend as much time as possible teaching K-12 students, we also found it important to use micro-teaching in our courses. One of the goals of our work as teacher educators was to narrow the gap between practice and theory, and micro teaching was a good pedagogical tool in this case.

Microteaching lesson in an out of doors setting.

The TEEMS program began each year as a summer institute, followed by two semesters of integrated school based teaching and learning and full-time internships. But on the first day of the summer institute each student in our program was asked to prepare a ten minute lesson on a science topic which would be taught in the next session to a small group of peers, or a small group of middle and high school students that we recruited from local districts. These sessions were not video taped, but a conference session was held among the teacher and “students” to discuss the lesson, and make suggestions for change. Another lesson was taught to different students incorporating any suggestions.

Learning to teach by teaching was an important goal of these early sessions, and set the stage for the further reconnaissance of science teaching.

Studying Teacher and Students in the Classroom

Researchers have used videos of teachers and students to study various aspects of teaching and learning for decades.

While I was a doctoral student at The Ohio State University (OSU) from 1966-1969, a main line of research was using video tape technology to study the classroom interactions and behavior of science teachers in Ohio schools. There were a dozen or more doctoral students involved in using the video data “inductively” to investigate science teaching. Our doctoral studies focused on secondary teachers in biology, chemistry, earth-science, and physics.

In my case I worked with Joe Abruscato, who also was also a doctoral student at OSU. Joe and I became life long friends and colleagues. We completed a joint study of teacher and student behavior in 20 earth science classrooms in several Ohio school districts. Each time we entered a classroom we lugged in two very large video cameras and tripods, and two video decks. We set up one camera facing towards a small group of students, and the other camera followed the teacher. Nothing back then was miniaturized, but we did have the “latest” technology. We figured out a way to make observations of teacher and student behavior simultaneously in the quest to correlate teacher and student behavior.

In our case, we developed teacher and student observation systems which we used in a follow-up study as part of our dissertation research. Although we looked at the relationship between the classroom behavior of teachers and the classroom behavior of students, are results were correlations, and there was no attempt to ask if certain teacher behaviors produced or caused student behavior.

We are on very shaky ground when we attempt to use reductionist thinking to ask questions about teaching. Today, the “corporate reformists” insist that student achievement can be predicted based on teacher’s “value added effect.” All of the research to support this idea is correlational, and very weak, limiting our conclusions to relational thinking, not cause and effect.

Teaching should be explored using video tape technology. But we should be cautious about how it is used, who decides when and how it is used, and the stakes that are on the table if video is used.

Interaction Analysis

In the 70s and 80s, one system of classroom observation was Interaction Analysis. In this use of video taping, teachers were observed using Interaction Analysis, which consisted of ten categories of classroom teacher and student behavior (teacher talk, student talk, silence). Video tapes were made of lessons and then observed by coding the behavior of the teacher and student (usually every 3 seconds). Teacher observations included accepts feelings, praises, accepts/uses student ideas, asks questions, lectures, gives directions, criticizes; student observations included pupil talk responding to the teacher, pupil talk that was initiated by the student. This system, and variants of it, were used for many years, especially in teacher education and staff development.

Interaction analysis is a behaviorist tool that many teachers, researchers, evaluators and administrators used to describe teaching. It was not an evaluation tool, but was symbolic of many inventories and systems to describe classroom practice.

In similar fashion, the current classroom observation system used in the Gates funded Measures of Effective Teaching (MET) study uses the Danielson Framework for Teaching, a 22 component system of teacher behavior and responsibility (with 76 smaller elements) in contrast to 10 categories in the Flanders system. The MET study results are used as one component to test teachers on the basis of these 22 categories, along with student achievement gains on high-stakes tests, and student surveys. You can check out the 22 categories of teacher performance here.

Video Tapes of Experienced Teachers

In this use of video tapes, novice teachers watch videos of experienced teachers. By watching an experienced teacher do the job, novice teachers would be able to pick up new ideas, and apply them in their own classrooms. A number of researchers modified this system by creating video tapes of teachers that was accompanied by commentary of the teacher. Novice teachers could reflect on the commentary, and hopefully participate in discussions of teaching and teaching practice. A good example is theAnnenberg Video Program. One series of videos that I’ve used and found useful is at the Annenberg Media Website of videos of science inquiry teaching. There is a large variety of inquiry lessons to observe at Annenberg. You can find some online video examples of science lessons here.

Reflection on Practice

Googling the phrase “using video for teachers to reflect on performance” results in more than 21,000 hits and that’s since 2009. Reflection on practice is a tool to use not only for self-reflection but also with trusted colleagues. Just as teaching is complex endeavor, reflection on practice is just as difficult and complex. Watching one or two videos will not lead to reflective practice. However, what is important is thinking deeply about one’s teaching practice, and seeing it as a fundamental part of professional teaching.

The Gates notion of video surveillance is not supported by the research on reflective practice. Reflective practice is based on a cognitive and social constructivist model of teaching, whereas Gates the approach is a behaviorist model that reduces teaching to a very long list of observable skills. Teaching is more of a system in which the gestalt of teaching is greater than the sum of the parts or a collection of skills. We should not evaluate teaching out-of-context. Uploading a video to trained video observer will only reinforce a behavioral model of teaching and degrade the efforts by professional teachers to improve their practice.

Video and Teacher Evaluation

The Bill and Melinda Gates foundation funded a three-year study, “Ensuring Fair and Reliable Measures of Effective Teaching,” which advocates three measures to evaluate teaching: classroom observation using measuring instruments, student perception surveys, and student achievement gains.

Each of these components has been the subject of criticism on the grounds of reliability and validity. There is a groundswell of protest about using high-stakes tests, with many administrators and teachers calling for a moratorium.

Jesse Rothstein of UC Berkeley in a review of Learning About Teaching, questions the results of the MET 2010 study which reported that a teacher’s estimated “value-added” computed from one year to the next was a measure of teaching effectiveness. Rothstein finds that the correlation between teacher’s value-added scores and student survey responses were only modestly related. Yet, the MET study claims that the value-added score is a valid part of teacher effectiveness.

The point here is that in the MET 2013 study, the third part, classroom observation scores was added to complete a three part system of teacher effectiveness. To suggest that the MET study has reliably and fairly produced measures of effective teaching is troubling. In Rothstein’s review of the of the 2010 MET study, the data in the study do not support the conclusions.

This is troubling because the Gates Foundation has used its money and influence to convince departments of education that the MET is a reliable and fair teacher evaluation system. Districts around the country will use value-added measures, student surveys, and the Danielson Classroom Observation System to evaluate teachers in a high-stakes environment.

The Gates Foundation study captured video of more than 23,000 lessons which were analysed using theDanielson Framework for Teaching (Framework). The Framework’s 22 components are assessed as unsatisfactory, basic, proficient, and distinguished. Even with this mass of data, if the measures used are not valid, then any conclusions that are drawn are suspect.

Using video to evaluate teacher performance is overly behaviorist, and reduces teaching to a set of skills that some trained observer looks for when viewing a video tape. Even though we know that breaking the art of teaching into discrete categories is reductionist, powerful forces have not only funded this teacher evaluation system, but many state departments have already employed it as the state teacher evaluation system.

According to Bill Gates (Ted talk), teacher evaluation and teacher improvement can go hand in hand. Here is how it would work:

What would that system look like? Well, to find out, our foundation has been working with 3,000 teachers in districts across the country on a project called Measures of Effective Teaching. We had observers watch videos of teachers in the classroom and rate how they did on a range of practices. For example, did they ask their students challenging questions? Did they find multiple ways to explain an idea? We also had students fill out surveys with questions like, “Does your teacher know when the class understands a lesson?” “Do you learn to correct your mistakes?”

And what we found is very exciting.

What causes students to learn is the question that the MET researchers claim to answer. They state, however that by definition teaching is effective when it enables student learning. According to the Gates study, effective teaching can be measured, and it was possible to state that teachers “caused” students to perform better.

What constitutes student learning? How is it observed and measured? Do students play any part or role in their own learning? Or is student learning, as suggested by MET researchers, caused by effective teachers.

Cameras in the classroom will not lead to the improvement of teaching, especially if they are used to evaluate teachers using a system is basically flawed from the beginning. What is disturbing to me is that states such as New York have mandated teacher evaluation using the frameworks that are MET-like, and that use the Danielson Framework. For example, the New York Commissioner of Education released a report outlining the three-tier evaluation system that will be used to evaluate NYC teachers. On the United Federation of Teachers website article the evaluation will be based on components that do not have the supporting research to put them into practice. The author of the article writes:

According to the 2010 state evaluation law, teachers will be rated Highly Effective, Effective, Developing or Ineffective with 20 points based on state measures of student learning, 20 points on local measures of student learning and 60 points on classroom observations and other measures of teacher performance.

Fortunately, in the NYC case, the AFT has taken an active role in working out details of the system that will support, not punish teachers. I am not sure.

What is your opinion of using video in classrooms as part of a teacher system that uses value-added, student surveys, and classroom observations?