In a previous article, we covered Step 1 of the Test Security Process by showing you how to determine which specific test security risks are the most significant for your individual program. The next step in the test security process, which this article will outline, is how you can develop a plan that focuses your priorities and resources on those risks particular to your program. Let’s get started.
Most exam programs don't think in terms of protecting an asset, but there is no better way to illustrate the purpose of test security. The most important asset for your testing program is your test scores. When trusted completely, these scores can be used for many valuable and important decisions (like whether a medical student is qualified to become a doctor or a student is prepared for the next grade level).
A secondary asset is the exams that are taken to produce those scores—it is important to make sure that those exams are not stolen or shared for the benefit of cheaters, which would negatively affect the assessment program’s most important asset.
Simply put, the key to a successful security plan for any testing program is to understand how to protect your assets from your threats—which are all laid out here.
The second recommendation is to embrace the Protection Process, commonly referred to as “the Test Security Process.” Protecting your two most important assets, your test scores and your exams, requires three classes of behaviors. Each class of behavior exists to counter the effectiveness of known test security threats within each category. The Protection Process involves (1) prevention, (2) deterrence, and (3) detection + reaction, in that order. You can think of it like this:
In order to have a well-rounded, sturdy, and holistic test security program, it's important to develop strategies and plans that cover all three categories of the Protection Process: strategies that (1) prevent, (2) deter, and (3) detect + react to threats. It is inevitable that unfortunately, sooner or later, breaches will occur. But when they do, you can be fully prepared to execute strong tactics within all three of these foundations, enabling you to keep your most important asset—your test scores—valid. Let's dive into each how each “leg” of the Protection Process works together to support a truly successful assessment program.
Prevention consists of actions a program can take to actually stop or reduce the risks associated with the threats before they even get off the ground. (An added benefit—while not its primary purpose, preventative solutions can have a significant deterrent effect as well when the scope and effectiveness of those solutions are communicated broadly.) Preventative measures are versatile and can (and should) be implemented at every stage of the assessment process—you can even design preventative test security measures directly into your exam questions (with innovative self-protecting items like these). Preventative measures include, but are not limited to:
The preventative strategies that your organization defines will flow nicely into the next set of strategies—deterrence. You can think of deterrence as a communication plan meant to discourage and inhibit test takers from cheating or committing other forms of test fraud. Deterrence is often just letting would-be test cheats and thieves know that there are consequences to their actions.
A good analogy for thinking about deterrence are the surveillance cameras used in retail stores. Through signage, businesses communicate their security measures that can catch shoplifters in the act, deterring would-be thieves from even attempting to steal merchandise. The same is true of communicating test security policies to test takers; by communicating the consequences of being caught cheating, you can deter individuals from ever attempting to commit fraud in the first place. (And, a plus side—when you utilize detection measures like data forensics and Web Patrol®, you can communicate that you have sophisticated test security measures in place that will catch cheating even after it has occurred! There’s no better deterrent than that.)
Another notable outcome of deterrence is that it lets individuals know what is and is not acceptable. Many examinees have a sincere desire to honestly take their exams. Having clear guidelines and communicating what is allowed before, during, and after the exam can help honest test takers avoid any “gray” areas when preparing to take their test, providing assistance to other test takers, or providing information about the test that shouldn’t be shared. (You can learn more about various deterrent measures—including proctoring—with these curated articles.)
Other examples of deterrence include (but are not limited to):
At this point, you’ve done your best to prevent test theft and cheating from occurring. However, how will you know when a breach has occurred? This is where the “detection + reaction” process comes in.
“But that’s two steps,” you say. While it can be viewed that way, it’s vital to mesh detection and reaction strategies into one smooth and seamless process. For example, if you uncover the very beginning of a test security breach (detect), that incident is associated with an immediate, specific, and almost automatic pre-determined set of responses (react). Those “automatic” responses have been strategically pre-planned and formulated to stop that specific type of breach from harming your test scores, and they have been custom-developed to quickly and efficiently mitigate any damages that have already occurred.
For example, let’s say your web monitoring team notifies you of a braindump site selling your live test questions. You’ve detected a test security incident. Luckily—since you’ve already developed an effective security incident response plan for this exact situation—your team can immediately respond without wasting any time. So, without a second thought, you immediately send a copyright infringement notice to the braindump site, remove and replace the affected items, and conduct data forensics to uncover exactly which test takers utilized the braindump content. With all of these pre-planned measures in place, you’ve 1) detected a security violation, 2) responded quickly, mitigating damage, 3) empowered your program to carry out ramifications, if desired, and 4) enabled you to evaluate the results of your detection + reaction process and determine whether changes should be made. But above all, you’ve protected your most important asset—the validity of your exam scores.
As with preventative measures, detection + reaction strategies can have a significant deterrent effect as well, but only if the scope and effectiveness of those solutions are broadly communicated (learn how you can do that in this article).
Another benefit of the detection + reaction process is its direct impact on the answers on which test security’s most important questions are framed: Are there any test takers that have gained an unfair advantage on my exam? What impact does this breach have on my exam’s performance? Can my test takers’ scores be trusted? Detection activities include, but are not limited to:
Of course, the actual reaction to the breach that your detection methods uncovered will vary greatly depending on the specific incident, so we won’t go into that here. But if you’re looking to get started on or fortify your incident response plan, these articles and this workbook are a great place to start.
Now you have come full circle. You have:
The last step is to repeat this process. Test security prevention is never fool-proof. Just when you think you’ve covered all your bases, someone will find a new way to cheat or steal your items. But don’t give up; keep fighting. Test security just needs a little love and a solid plan. By thinking constructively and working diligently, you can protect your two most important assets—your test scores and the exams that produce those scores—from the many threats that are present. Remember that each “leg” of the Protection Process—prevention, deterrence, and detection + reaction—must work together to support and secure your assessments and keep your exam results fair and valid. When usable, the results that your test produces can be trusted for valuable and important decisions.
Topics from this blog: Test Security Consulting Data Forensics Exam Development Investigating Security Incidents K-12 Education Web Monitoring Test Security Basics Monitoring Test Administration DOMC™ Detection Measures Deterrence Measures Prevention Measures Certification Test Security Plan Higher Education Online Exams SmartItem™ Security Incident Response Plan Medical Licensure
CAVEON CAN HELP YOU COVER ALL THREE STAGES OF THE PROTECTION PROCESS
