The World's Only Test Security Blog
Pull up a chair among Caveon's experts in psychometrics, psychology, data science, test security, law, education, and oh-so-many other fields and join in the conversation about all things test security.
Posted by John Olson, Ph.D.
updated over a week ago
With COVID-19 suddenly impacting K-12 educational systems across the country, testing in Spring 2020 offered many new challenges and uncertainties in the form of a pandemic, which ultimately led to the cancellation of statewide assessments. States had to pivot quickly to come up with plans for conducting required assessments in the coming year. In addition, with distance/virtual learning and remote testing increasingly being implemented in many states and school districts, the security of assessments came into question, along with how to maintain the validity and integrity of the test scores. In the 2020-21 school year, many new approaches to testing had to be designed, developed, and implemented by states.
The Spring of 2021 moved statewide testing forward but required State Education Agencies (SEAs) across the country to adopt new policies and procedures to allow assessments to occur remotely (such as in student homes) or in unconventional settings within schools. Simply not testing again was not an option, since statewide testing is federally mandated under the Every Student Succeeds Act (ESSA) and states were also required to meet all Peer Review requirements set forth by the U.S. Department of Education. For the most part, the USED did not grant any waivers from testing in 2021 (except in D.C.), although they allowed all states to drop the accountability requirements for that year.
Over the past 20 months, a large amount of data has been gathered from states on how testing occurred, or not, during the ongoing COVID pandemic in 2020-21. As part of my work with the Assessment Solutions Group, State Assessment Directors (SADs) were surveyed in Fall 2020 regarding the assessment approaches they planned to use in 2021 and what they were going to do to maintain the security of their assessments. Data from that survey highlighted many things being considered by states—more use of diagnostic, interim, through-course, and formative approaches, with an emphasis on student learning, along with summative assessments. Many issues and challenges were identified, such as measuring learning loss in 2020 and 2021, opportunity to learn, connectivity, equity, comparability of results across years, and maintaining test security and the validity of test scores.
In addition to these survey findings, other sources of data on what states did in 2021 to successfully administer their assessments during the pandemic come from recent presentations made at the National Conference on Student Assessment (NCSA) and the Conference on Test Security (COTS), Technical Issues in Large Scale Assessment (TILSA) meetings, other recent surveys of State Assessment Directors (SADs), and interactions between Caveon staff and many state assessment staff. Based on information gathered from these various sources over the past year, some of the changes that states have made in their testing programs include:
Many new challenges and lessons were learned from those states that securely administered tests in 2020-21 during the continuing pandemic. Among the many challenges were conducting secure testing in remote environments and the use of remote proctors or record-and-review approaches, the use of data forensics analyses to detect possible cheating, increased monitoring of the internet and social media for disclosed assessment materials, ensuring students received proper testing accommodations under all conditions of testing, and taking extra steps to ensure that scores from statewide summative and alternate assessments were validated.
Based on what was learned by states and others over the past year, some possible directions states may be going in future years include:
Despite which directions states may go with their assessments, in the coming year(s), it will be critically important for states to maintain a focus on ensuring test security throughout the entire assessment process, including in the design, development, administration, scoring, analysis, and reporting for tests. This involves the possible use of innovative test items and approaches that have built-in security, stronger training procedures for all test administrators, conducting remote testing virtually in a secure environment, better monitoring of all types of test administrations, and applying best practices in test security for every component of the state assessment program.
Dr. John F. Olson is President of the consulting business he founded in 2006, Olson Educational Measurement & Assessment Services (OEMAS), which provides technical assistance and support to states, school districts, the U.S. Department of Education, the National Assessment of Educational Progress (NAEP), Ministries of Education in other countries, CCSSO, Caveon Test Security, testing companies, researchers, and others. He has more than 30 years of experience providing consulting on a variety of measurement and statistical issues for international, national, state, and local assessment programs. Dr. Olson also currently serves as senior partner for the Assessment Solutions Group (ASG), which he co-founded in 2009. The mission of ASG is to help states and local districts maximize value throughout the assessment procurement and implementation process via service offerings in RFP preparation, bid analysis and proposal evaluation, cost analysis, price negotiations, and ongoing program and contract management. He has worked with many states, CCSSO, SCOPE, and many of the large common assessment consortia, including PARCC, SBAC, NCSC, and WIDA ASSETS. Previously, he served as Vice President for Psychometrics and Research Services at Harcourt Assessment, Director of Assessment for CCSSO and the SCASS projects, Deputy Director of the Center for Education Assessment at American Institutes for Research (AIR), Senior Research Scientist with the Education Statistics Services Institute (ESSI), and in a number of leadership roles for NAEP at the Educational Testing Service (ETS). Olson holds a Ph.D. in educational statistics and measurement from the University of Nebraska - Lincoln. His specialties include educational assessment, measurement, and statistics; state assessment programs; national and international assessments, including NAEP and TIMSS; psychometrics; management; research; technical and report writing; RFP writing; technical evaluations; and business strategies.View all articles
For more than 18 years, Caveon Test Security has driven the discussion and practice of exam security in the testing industry. Today, as the recognized leader in the field, we have expanded our offerings to encompass innovative solutions and technologies that provide comprehensive protection: Solutions designed to detect, deter, and even prevent test fraud.
Topics from this blog: K-12 Education