What Are the U.S. Department of Education's Peer Review Guidelines?

What Are the U.S. Department of Education's Peer Review Guidelines for K-12 Schools?

The U.S. Department of Education (USED) Assessment Peer Review Guidelines for K-12 schools are a set of federal requirements for state assessments. These guidelines exist to ensure that state-wide tests are fair and secure and that test scores are accurate, valid, and reliable.

There are many different sections within the federal Assessment Peer Review guidelines. You can view USED's website here for more information, or read the latest iteration of the Peer Review Guidelines in this document from USED. The critical elements for test security were added to the Peer Review Guidelines in 2018. This article will focus specifically on those Test Security Peer Review Requirements. They are:

• Critical Element 2.4—Monitoring Test Administration
• Critical Element 2.5—Test Security
• Critical Element 5.4—Monitoring Test Administration for Special Populations

The above three elements contain the validity evidence that states are required to collect in order to indicate sufficient test security procedures and practices are in place. Here's how they fit in to the overall Peer Review Guidelines (USED, 2018, p. 29):

What Are the Test Security-Specific Assessment Guidelines within the Federal Peer Review Requirements?

Critical Element 2.4—Monitoring Test Administration

Critical Element 2.4 of the U.S. Department of Education’s Peer Review Guidelines states that you must assemble a site monitoring plan:

“The State adequately monitors the administration of its State assessments to ensure that standardized test administration procedures are implemented with fidelity across districts and schools. Monitoring of test administration should be demonstrated for all assessments in the State system: the general academic assessments and the AA-AAAS.”

A site monitoring plan helps USED know whether test security policies are being implemented effectively. Many programs work with a vendor to meet this Peer Review Requirement due to the complexity of site monitoring strategies and test administration guidelines. This is especially helpful for programs that may not have the experience to know whether best practices are being met or whether they are making the most efficient use of their resources. In many cases, teaming up with a test security company that specializes in quality site monitoring programs is a seamless way to ensure this peer review requirement is met, and that your assessments are being securely administered.

If you do decide to team up with a test security expert for Critical Element 2.4, try to determine what form of site monitoring would best help your program. There are a range of site monitoring services that can be tailored to your program. For example, one program might use virtual site monitoring training to coach their own team on test administration best practices, and another might utilize a full-scale on-site monitoring service that provides all the tools needed to meet Peer Review Requirements. When deciding the best route for your program, consider your budget, personnel, and security needs. View these resources for more information: FOR STATES—The Ultimate Guide to Site Monitoring and FOR DISTRICTS: The Ultimate Guide to Monitoring Test Administration. We also have a white paper that discusses test security and peer review language for state assessment RFPs.

Critical Element 2.5—Test Security

Critical Element 2.5 of USED’s Assessment Peer Review Requirements states that you must have policies and procedures that prevent testing irregularities:

“The State has implemented and documented an appropriate set of policies and procedures to prevent test irregularities and ensure the integrity of test results through:

• • • “Prevention of any assessment irregularities, including maintaining the security of test materials (both during test development and at time of test administration), proper test preparation guidelines and administration procedures, incident-reporting procedures, consequences for confirmed violations of test security, and requirements for annual training at the district and school levels for all individuals involved in test administration;
    • “Detection of test irregularities;
    • “Remediation following any test security incidents involving any of the State’s assessments;
    • “Investigation of alleged or factual test irregularities.
    • “Application of test security procedures to all assessments in the State system: the general academic assessments, the general ELP assessments, the AA-AAAS, and the AELPA.”

USED provides examples of evidence states may provide, such as state test security handbooks and "summary results or reports of internal or independent monitoring, audit, or evaluation of the state's test security policies, procedures, and practices" (USED, 2018, p. 43). With that said, Critical Element 2.5 is most commonly tag-teamed with external test security experts to help with things like investigations. But help can range based on a program’s needs. For example:

• • One program might opt for yearly professional training for their in-house testing staff and enhanced technology that's specifically made for tracking and managing incidents.
  • Another program might opt for a comprehensive investigation solution like this one (meaning that when a breach is suspected, your vendor deploys a team of experienced test security investigators to take care of the whole thing for you—everything from conducting an independent investigation, collecting, witness accounts, managing incident reports, leading interviews, and more). 
  • Another program might use data forensics to detect testing irregularities.

Each of these example options addresses the evidence states can provide to support Critical Element 2.5. In all, when deciding the best route for your program, consider your budget, personnel, and security needs. And, when needed, reach out for help. There are test security experts who can provide guidance for your specific situation.

Critical Element 5.4—Monitoring Test Administration for Special Populations

Critical Element 5.4 of the Assessment Peer Review Requirements has been getting a lot of attention from USED in recent years, since many states are not doing enough to monitor the administration of alternative assessments or testing of SWDs and ELs. The assessment guidelines state that:

"The State monitors test administration in its districts and schools to ensure that appropriate assessments, with or without accommodations, are selected for all students with disabilities and ELs so that they are appropriately included in assessments and receive accommodations that are:

• • • "Consistent with the State’s policies for accommodations;
    • "Appropriate for addressing a student’s disability or language needs for each assessment administered;
    • "Consistent with accommodations provided to the students during instruction and/or practice;
    • "Consistent with the assessment accommodations identified by a student’s IEP Team under IDEA, placement team convened under Section 504; or for students covered by Title II of the ADA, the individual or team designated by a district to make these decisions; or another process for an EL;
    • "Administered with fidelity to test administration procedures;
    • "Monitored for administrations of all required academic content assessments, AA-AAAS, ELP assessments, and AELPA."

For Critical Element 5.4, you'll need to provide 1) proof of procedures that the state uses to ensure special populations of students are placed by individuals or teams designated by a district to make such decisions; 2) documentation of procedures that demonstrate the monitoring of accommodations selected for students with disabilities and ELs are appropriate; and 3) results of monitoring from the most recent year of test administration in your state. You can view these resources for more information: FOR STATES—The Ultimate Guide to Site Monitoring and FOR DISTRICTS: The Ultimate Guide to Monitoring Test Administration. 

Conclusion

Critical Elements 2.4, 2.5, and 5.4 of the U.S. Department of Education's Assessment Peer Review Requirements contain the validity evidence that states are required to collect to indicate sufficient test security procedures and practices. You can work with a test security expert for either (or all three) of them, but make sure your vendor has experience working with Peer Review Guidelines and implementing test security best practices into K-12 assessments. No matter your program’s budget, personnel, or security needs, your vendor should always be able to work with you to create a strategy that works best for you and your program.