BOULDER, CO (March 5, 2019) – Between May and August of 2018, the federal government approved 44 proposals submitted by state departments of education to meet testing and accountability requirements under the Every Student Succeeds Act (ESSA) of 2015. As these states move towards implementing their federally approved plans for meeting external regulatory requirements for accountability, they face several challenges. They should be alert to any unexpected negative outcomes of testing on students, schools, and education systems, and they need guidelines to help them avoid test misuses. These guidelines should explain how best to apply information from the adopted tests with a mind to their design and with close attention to their purposes, technical merits and limitations.
Today, NEPC is releasing A Consumer’s Guide to Testing under the Every Student Succeeds Act (ESSA): What Can the Common Core and Other ESSA Assessments Tell Us?, written by Professor Madhabi Chatterji of Teachers College, Columbia University.
As Professor Chatterji explains, misuse of test information in educational accountability contexts is like misreading a Fahrenheit thermometer in degrees Celsius. Such misuses can pose unexpected barriers to goals that educators and stakeholders hope to achieve in schools. Her guide analyzes current conditions in the context of ESSA—conditions that could cause serious test misuses similar to those of the past.
Her analysis finds that new testing programs, ESSA’s latest accountability requirements, and state plans combine to present “black boxes” that increase the chances for improper test uses and negative consequences for students, teachers and schools. She provides guidelines that could improve uses and transparency of test information in school rating and ranking systems that states and other educational entities plan to use in their accountability programs. The guide focuses on standardized achievement tests, other academic and non-academic measures, and combined quality indices that rate or rank schools.
A well-designed educational test is a useful tool, but it can only do so much. Under ESSA, however, many education systems are making multiple demands on a single test without due attention to its limitations, and several have proposed uses of test information at student- and upper-levels of the system without sufficient evidentiary support of validity, reliability and utility in hand. These are “Red Flags” that test users and test makers should heed, as they could lead to inadvertent test-based misinterpretations and misuses of information.
To preempt inappropriate or unjustified inferences and uses with test-based information, Professor Chatterji recommends that test users (a) specify all intended test-based inferences and uses up front; (b) avoid multi-purposing a test in ways that exceed a test’s declared purposes or reported evidence; (c) justify all planned inferences and uses of test-based data using appropriate criteria for validity, reliability and utility (see inside for definitions); and (d) seek out expert technical reviews of tests and non-academic measures before adopting these tools for accountability purposes. Professor Chatterji also offers seven additional concrete recommendations for policy makers, test makers, and education stakeholders involved in implementing accountability programs in education systems.
Find A Consumer’s Guide to Testing under the Every Student Succeeds Act (ESSA): What Can the Common Core and Other ESSA Assessments Tell Us?, by Madhabi Chatterji, at:
https://nepc.colorado.edu/publication/rd-assessment-guide