BOULDER, CO (January 20, 2022) – In this month’s episode of NEPC Talks Education, NEPC Researcher Christopher Saldaña interviews Scott Marion and Derek Briggs about the reporting and use of state and national test results. Marion is the Executive Director of the Center for Assessment and a member of the National Assessment Governing Board. Briggs is a professor in the Research and Evaluation methodology program in the School of Education at the University of Colorado Boulder. He is also the current president of the National Council on Measurement in Education. His new book is Historical and Conceptual Foundations of Measurement in the Human Sciences: Credos and Controversies.
Briggs and Marion explain that some testing experts were worried that states would vary in their capacity to administer tests during the pandemic, that changes to the testing population caused by the pandemic would make interpreting results difficult, and that instructional time might be better spent learning instead of testing. Now that tests have been administered and state results are in, they argue that in spite of challenges, we learned a lot about the effects of the pandemic on student achievement thanks to many researchers and technical advisors putting forth methods to account—to the extent possible—for the challenges outlined above.
They explain that best practices included contextualizing 2020-21 test results by describing how enrollment changed during the pandemic and identifying which students were the most likely/unlikely to participate in testing (e.g., economically disadvantaged, special education, or bilingual students). Doing this allowed policymakers and the public to better understand how the pandemic influenced the reported results. Briggs adds that if state policymakers failed to provide adequate context, users might have underestimated the impact of the pandemic on learning. He explains that average test scores could be biased upwards by the disproportionate absence of students with the greatest needs.
Briggs and Marion also comment on the recent release of results of the National Assessment of Educational Progress. Unlike state results, they explain that NAEP is a national indicator of both national and state trends in public education. Briggs explains that the most recent release of NAEP results shows that the gap in academic performance between the lowest and highest achieving students in the United States is widening. Marion adds that while the larger gaps should be addressed, the causes of these gaps in performance are a matter of dispute. Some analysts, for example, view the performance gaps as primarily attributable to social and economic factors that have resulted in fewer resources for schools and growing wealth inequality, while critics of the education reforms of the past two decades such as common core curricula could argue these educational changes are to blame for the observed gaps in test results. Marion and Briggs note this is the perfect example of why NAEP is so useful, but also why its use is limited. While it provides a stable and high-quality measure of educational trends in the achievement among American students, it cannot explain the causes of those trends.
Briggs and Marion believe several practices that emerged during the pandemic should be continued. Marion describes how early in the pandemic, district leaders collected data on student needs. He encourages policymakers to continue that collection and supplement it by collecting data, such as whether students were instructed on certain concepts and whether they received regular feedback, that indicate students’ opportunities to learn (OTL). He argues test score data is made more useful when it can be paired with high-quality OTL data. Briggs adds that the federal government and states should learn from the flexibility that was required during the pandemic. He noted that disruptions are likely to continue, and federal, state, and local policymakers will need to be ready. Both Briggs and Marion recommend that policymakers respond aggressively to provide support for schools in the years to come. They explain that there is overwhelming evidence to show that students’ educational opportunities were severely diminished during the pandemic. They argue that helping students overcome these setbacks will require a historic investment and effort in K-12 public schools.
A new NEPC Talks Education podcast episode, hosted by Christopher Saldaña, will be released each month from September through May.
Don’t worry if you miss a month. All episodes are archived on the NEPC website and can be found here.
NEPC podcast episodes are also available on Apple Podcasts, Spotify, and Stitcher, under the title NEPC Talks Education. Subscribe and follow!