Key Review Takeaway: Weak and poorly presented study renders report’s conclusions about Common Core and testing unjustified
Contact:
William J. Mathis: (802) 383-0058, wmathis@sover.net
Sharon L. Nichols: (210) 458-2035, Sharon.Nichols@utsa.edu
Find Documents:
Press release: http://tinyurl.com/hadqv2z
Review: http://nepc.colorado.edu/thinktank/review-CAP-standards
Report reviewed: https://cdn.americanprogress.org/wp-content/uploads/2015/12/23090515/NAEPandCommonCore.pdf
BOULDER, CO (February 25, 2016) – A recent report from the Center for American Progress (CAP) examines the standards-based policies that have dominated the national landscape over the last few decades. The report explores whether states’ adoption of standards-based policies predicts low-income students’ achievement trends in fourth and eighth grade math and reading from 2003 to 2013.
A review of the report finds that it employs inappropriate research methods, fails to adequately define its approach, and reports only incomplete findings from its analyses. The report’s claims far outpace its meager evidence.
Sharon L. Nichols, Associate Professor of Educational Psychology at the University of Texas at San Antonio, reviewed Lessons From State Performance on NAEP: Why Some High-Poverty Students Score Better Than Others for the Think Twice Think Tank Review Project at the National Education Policy Center, housed at the University of Colorado Boulder’s School of Education.
The report, according to Professor Nichols’ review, does not adequately describe variables or analytic methods, and the data and methods used do not allow for any causal findings. Yet the report uses causal language such as, “According to our analysis, states typically saw a jump in outcomes due to standards-based reform …” (emphasis added). The data and methods used, even if the associations were much stronger than those shown in this study, could not possibly support such conclusions.
Also, while the report claims to analyze changes across five separate two-year intervals, it only reports findings for 2009-2011, with no explanation of why or any documentation of the representativeness of that single interval.
The positive results the report does find, Professor Nichols points out, are statistically significant only at the generally unacceptable 0.10 level of significance. Further, while the report includes effect sizes, it does not report the percentage of the variance explained in the model.
The report contends that its analysis strongly supports the policy recommendation that states “embrace” the Common Core and its aligned assessments. But the study is far too weak to do this. As Professor Nichols explains, “Even if the omissions and shortcomings of this report were remedied, the analysis only provides a very narrow snapshot of how policy might connect to practice. … The strident call for Common Core at the end of this report is misplaced given [its] mismatched goals, questionable analysis and selected findings.”
Find Professor Nichols’ review at:
http://nepc.colorado.edu/thinktank/review-CAP-standards
Find Lessons From State Performance on NAEP: Why Some High-Poverty Students Score Better Than Others, by Ulrich Boser and Catherine Brown, published by the Center for American Progress, at:
https://cdn.americanprogress.org/wp-content/uploads/2015/12/23090515/NAEPandCommonCore.pdf
The National Education Policy Center (NEPC) Think Twice Think Tank Review Project (http://thinktankreview.org) provides the public, policymakers, and the press with timely, academically sound reviews of selected publications. The project is made possible in part by support provided by the Great Lakes Center for Education Research and Practice: http://www.greatlakescenter.org/
The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at: http://nepc.colorado.edu/