I first asked this question in a post published over six years ago. I have updated and revised that post because the answer is popularly and resoundingly “yes” although the evidence is squirmy.  I revisit both the question and answer.

After Atlanta (GA) school administrators and teachers went to trial and were convicted and sentenced to jail for cheating and before that the El Paso (TX) superintendent convicted of the same charge and in prison, the generally accepted idea that district superintendents can pump up student  achievement has taken a serious hit. Cheating scandals across the country have turned the belief in superintendents raising test scores into something tawdry.

For decades, many superintendents have been touted as earnest instructional leaders, expert managers, and superb politicians who can mobilize communities and teacher corps to improve schools and show gains in students’ test scores. From Arlene Ackerman in Philadelphia to Joel Klein in New York City to Kaya Henderson in Washington, D.C., big city superintendents are at the top rung of those who can turn around failing districts.

Surely the Atlanta cheating scandal and others around the country have tarnished the image of dynamic superintendents taking urban schools from being in dumpsters to $1 million Broad Prize winners. A tainted image, however, will not weaken the Velcro belief that smart district superintendents will lead districts to higher student achievement. Just look at contracts that school boards and mayors sign with new superintendents. Contract clauses call for student test scores, graduation rates, and other academic measures to increase during the school chief’s tenure (see here and here).

Then along comes a study that asks whether superintendents are “vital or irrelevant.” Drawing on state student achievement data from North Carolina and Florida for the years 1998-2009, researchers sought to find out how much of a relationship existed between the arrival of new superintendents, how long they served, and student achievement in districts (see PDF SuperintendentsBrown Center9314).

Here is what the researchers found:

1. School district superintendent is largely a short-term job. The typical superintendent has been in the job for three to four years. 
2. Student achievement does not improve with longevity of superintendent service within their districts. 
3. Hiring a new superintendent is not associated with higher student achievement. 
4. Superintendents account for a small fraction of a percent (0.3 percent) of student differences in achievement. This effect, while statistically significant, is orders of magnitude smaller than that associated with any other major component of the education system, including: measured and unmeasured student characteristics; teachers; schools; and districts. 
5. Individual superintendents who have an exceptional impact on student achievement cannot be reliably identified.

Results, of course, are from only one study and must be handled with care. The familiar cautions about the limits of the data and methodology are there. What is remarkable, however, is that the iron-clad belief that superintendents make a difference in student outcomes held by the American Association of School Administrators, school boards, and superintendents themselves has seldom undergone careful scrutiny. Yes, the above study is correlational. It does not get into the black box of exactly how and what superintendents do improves student achievement.

Ask superintendents how they get scores or graduation rates to go up.  The question is often answered with a wink or a shrug of the shoulders. Among most researchers and administrators who write and grapple with this question of whether superintendents can improve test scores, there is no explicit model of effectiveness. That is correct, there is no theory of change, no theory of action.

How exactly does a school chief who is completely dependent on an elected school board, district office staff, a cadre of principals whom he or she may see monthly, and teachers who shut their doors once class begins–raise test scores, decrease dropouts, and increase college attendance? Without some theory by which a superintendent can be shown to have causal effects, test scores going up or down remain a mystery or a matter of luck that the results occurred during that school chief’s tenure (I exclude cheating episodes where superintendents have been directly involved because they have been rare).

Many school chiefs, of course, believe–a belief is a covert theory–that they can improve student achievement. They hold dear the Rambo model of superintending. Strong leader + clear reform plan + swift reorganization + urgent mandates + crisp incentives and penalties =  desired student outcomes. Think former New York City Chancellor Joel Klein, ex-Miami-Dade Superintendent Rudy Crew, ex-Chancellor of Washington D.C.and ex-school chief Alan Bersin in San Diego. Don’t forget John Deasy in Los Angeles Unified School District. And now, Pedro Martinez in San Antonio Independent School District

There are, of course, other less heroic models or theories of action that mirror more accurately the complex, entangled world of moving school board policy to classroom practice. One model, for example, depicts stable, ongoing, indirect influence where superintendents slowly shape a district culture of improvement, work on curriculum and instruction, insure that  principals run schools consistent with district goals, support and prod teachers to take on new classroom challenges, and communicate often with parents about what’s happening. Think ex-superintendents Carl Cohn in Long Beach (CA), Tom Payzant in Boston (MA) and Laura Schwalm in Garden Grove (CA). Such an indirect approach is less heroic, takes a decade or more, and ratchets down the expectation that superintendents be Supermen or Wonder Women.

Whether school chiefs or their boards have a Rambo model, one of indirect influences, or other models, some theory exists to explain how they go about improving student performance. Without some compelling explanation for how they influence district office administrators, principals, teachers, and students to perform better than they have, most school chiefs have to figure out their own personal cause-effect model, rely upon chance, or even in those rare occasions, cheat.

What is needed is a crisp GPS navigation system imprinted in school board members’ and superintendents’ heads that contain the following:

*A map of the political, managerial, and instructional roles superintendents perform, public schools’ competing purposes, and the constant political responsiveness of school boards to constituencies that inevitably create persistent conflicts.

*a clear cause-effect model of how superintendents directly influence principals and teachers and they, in turn,influence students to do better as in creating incentives and sanctions, a culture of trust that encourages both risk-taking and willingness to learn.

*a practical and public definition of what constitutes success for school boards, superintendents, principals, teachers, and students beyond standardized test scores, higher graduation rates, and college admissions.

Such a navigation system and map are steps in the right direction of answering the question of whether superintendents can raise test scores.