Conducting State Assessments During the Pandemic: Continuing Challenges and Ongoing Changes in Maintaining & Improving Test Security

Challenges for State Assessment Programs

With COVID-19 suddenly impacting K-12 educational systems across the country, testing in Spring 2020 offered many new challenges and uncertainties in the form of a pandemic, which ultimately led to the cancellation of statewide assessments. States had to pivot quickly to come up with plans for conducting required assessments in the coming year. In addition, with distance/virtual learning and remote testing increasingly being implemented in many states and school districts, the security of assessments came into question, along with how to maintain the validity and integrity of the test scores. In the 2020-21 school year, many new approaches to testing had to be designed, developed, and implemented by states.

The Spring of 2021 moved statewide testing forward but required State Education Agencies (SEAs) across the country to adopt new policies and procedures to allow assessments to occur remotely (such as in student homes) or in unconventional settings within schools. Simply not testing again was not an option, since statewide testing is federally mandated under the Every Student Succeeds Act (ESSA) and states were also required to meet all Peer Review requirements set forth by the U.S. Department of Education. For the most part, the USED did not grant any waivers from testing in 2021 (except in D.C.), although they allowed all states to drop the accountability requirements for that year.

Over the past 20 months, a large amount of data has been gathered from states on how testing occurred, or not, during the ongoing COVID pandemic in 2020-21. As part of my work with the Assessment Solutions Group, State Assessment Directors (SADs) were surveyed in Fall 2020 regarding the assessment approaches they planned to use in 2021 and what they were going to do to maintain the security of their assessments. Data from that survey highlighted many things being considered by states—more use of diagnostic, interim, through-course, and formative approaches, with an emphasis on student learning, along with summative assessments. Many issues and challenges were identified, such as measuring learning loss in 2020 and 2021, opportunity to learn, connectivity, equity, comparability of results across years, and maintaining test security and the validity of test scores.

Ongoing Changes to State Assessments

In addition to these survey findings, other sources of data on what states did in 2021 to successfully administer their assessments during the pandemic come from recent presentations made at the National Conference on Student Assessment (NCSA) and the Conference on Test Security (COTS), Technical Issues in Large Scale Assessment (TILSA) meetings, other recent surveys of State Assessment Directors (SADs), and interactions between Caveon staff and many state assessment staff. Based on information gathered from these various sources over the past year, some of the changes that states have made in their testing programs include:

• Shortening the length of their tests by reducing the number of items or eliminating performance tasks
• Using longer testing windows to give districts and schools more time to complete all test administrations
• Delaying or canceling some test administrations
• Providing options for conducting testing in school settings on weekends or evenings
• Moving toward the implementation of remote testing, which was done in a handful of states
• Responding to last-minute requests from key policymakers and stakeholders in the state, such as the Commissioner or State Legislators, to implement new approaches for testing

Many new challenges and lessons were learned from those states that securely administered tests in 2020-21 during the continuing pandemic. Among the many challenges were conducting secure testing in remote environments and the use of remote proctors or record-and-review approaches, the use of data forensics analyses to detect possible cheating, increased monitoring of the internet and social media for disclosed assessment materials, ensuring students received proper testing accommodations under all conditions of testing, and taking extra steps to ensure that scores from statewide summative and alternate assessments were validated.

Possible Future Directions for State Assessments

Based on what was learned by states and others over the past year, some possible directions states may be going in future years include:

• Implementing more balanced assessments that include a combination of summative, interim, and/or formative assessments
• Use of more hybrid approaches and allowing increased flexibility for districts to use tests of their choosing
• Administering most, if not all, assessments online
• Use of CATs and Multi-stage adaptive assessments
• Increased use of interim/benchmark and through-course assessments, with the goal of possibly replacing the summative assessment with these approaches
• Serious consideration of remote testing as an ongoing option
• Use of virtual and other types of monitoring for remote test administrations
• More use of web monitoring procedures to check for disclosed test materials on the Internet and social media platforms
• Ongoing use of data forensics analyses to check for testing irregularities and anomalous test results
• Increased emphasis on equity, fairness, comparability, validity, reliability, and testing integrity for all assessment components, not just the summative test

Despite which directions states may go with their assessments, in the coming year(s), it will be critically important for states to maintain a focus on ensuring test security throughout the entire assessment process, including in the design, development, administration, scoring, analysis, and reporting for tests. This involves the possible use of innovative test items and approaches that have built-in security, stronger training procedures for all test administrators, conducting remote testing virtually in a secure environment, better monitoring of all types of test administrations, and applying best practices in test security for every component of the state assessment program.