BOULDER, CO (November 7, 2024)—A recent report from the Center for Education Policy Analysis at the University of Colorado Denver attempts to assess the effect of a “portfolio district strategy” on student performance in the Denver Public Schools. As was the case with its 2022 predecessor (reviewed here), advocates for portfolio reforms have trumpeted the new study as showing that this “portfolio” package of reforms has a positive and large impact on student outcomes. The review, however, concludes that the study lacks the ability to so definitively link Denver’s improved outcomes to these particular reforms.

In his review, Robert Shand of American University finds that the assertions in this year’s report, titled Systemwide and Intervention-Specific Effects of Denver Public Schools’ Portfolio District Strategy on Individual Student Achievement, are exaggerated in both magnitude and certainty.

The current report seriously addresses Professor Shand’s previous criticism that the 2022 study had potentially misattributed academic gains to the portfolio reform by inadequately addressing the alternative explanation that student demographics that had changed during the reform period. The new report convincingly demonstrates that the gains are not primarily due to changing demographics.

But the 2024 report fails to address Professor Shand’s other critiques of the prior study, including (1) that the mechanisms whereby the portfolio model might improve outcomes (and not include possibly harmful elements) are undertheorized; and (2) that things besides what would be typically understood as included within the portfolio reform (or student demographics) were changing concurrently with the reform. As the review points out:

The report dismisses factors such as additional funding, reduced class sizes, and changes in leadership or curriculum concomitant with the reform. As just one example, DPS received a large infusion of external resources during the time period under study, including over $15 million raised in 2014-15, the last year of the study period. This equates to about $175 per student from just one source.

In sum, while these researchers and others have indeed observed significant academic improvements in Denver, the report’s exuberant conclusion that Denver’s reform is “among the most effective” in U.S. history cannot be supported by the study. In fact, the study—while it effectively ruled out demographic change—cannot identify the causes of the improved outcomes.

Find the review by Robert Shand at:
https://nepc.colorado.edu/review/denver

Find Systemwide and Intervention-Specific Effects of Denver Public Schools’ Portfolio District Strategy on Individual Student Achievement, written by Parker Baxter, Anna Nicotera, David Stuit, Margot Plotz, Todd Ely, and Paul Teske and published by Center for Education Policy Analysis at the University of Colorado Denver, at: https://publicaffairs.ucdenver.edu/docs/librariesprovider36/default-document-library/denver_systemwide_and_intervention-effects-technical-report-september-2024.pdf