There is comfort and security that come from following a reliable road map to a successful end. It has been my experience that successful people don't make things up as they go along, but have learned and watched from others and follow a proven path. The process outlined in this article is one such roadmap, and it will help anyone in charge of test security, or anyone responsible for caring for tests and the scores they produce.
At Caveon, it has taken us years to develop and refine an exceptional test security process, and we want to share it with you. Within each step, you will find articles and other resources that help you complete each respective step. Whether you are in charge of the security of exams, play another role in your testing program, or are simply affected by good and bad security decisions, this 4-step process will be a valuable roadmap for you. Take it, use it, and pass it along to others you know it would help.
The first step is to determine which specific test security risks are the most significant to your program. The most important asset for your testing program is your test scores. You should be able to recognize the security threats specific to your test, calculate each threat's risk to your program, and rank them (you can learn how in this article). This first vital step will help make sure that you are tackling the risks that are the most likely to threaten the validity of your exam. Once you have determined your program's unique risks, you can allocate and budget resources to deal with the highest priorities identified.
Example: Say you have found out that 1) some instructors, who are also aiding in proctoring the exams, are helping their students (and your test takers) cheat. This, you decide, is the threat that carries the most significant risk to your test today. Then, as a second threat to your program, you are also very worried that 2) students have been taking pictures of the test screens and sharing them over social media. After calculating these two threats' risks to your program, you determine that both of these threats are where you should focus and prioritize your security resources. (Keep this example in mind, we'll reference these two identified threats as examples in Step 2.)
This step has three critical sub-steps, each of which should be equally and seriously considered. These steps are prevention, deterrence, and detection + reaction. The Protection Process is briefly summarized below, but we highly recommend reading more about it in this in-depth article.
Here you put in place security measures that actually make test fraud impossible or significantly more difficult to carry out.
Example: Prevention measures that could combat our two example threats include 1) changing your test administration approach so that tests are monitored by online proctors, thereby helping to eliminate the possibility that instructors are helping students cheat and 2) implementing a policy where cell phones are gathered up before testing begins and where tests are designed to more effectively limit the exposure of item content.
Each security measure you take here will help the test taker decide that they do not want to cheat on your exam or steal and distribute your test content. Remember that deterrence measures are only effective if they are made known, which means they should be announced and published (more on publicizing your test security in this article).
Example: Deterrence measures for our two example threats include 1) publishing a policy that any instructor caught helping a student cheat on a test will be fired, or 2) announcing that the new design of your tests will make it difficult to harvest questions. As a third measure, even having students sign a non-disclosure agreement before taking the test will have an impactful deterrent effect.
Every great defense has multiple ways of telling whether an attack is imminent or already underway, as well as a strategy to counter the attack (think of both of these as one swift movement). In the world of testing, a program manager should employ a sensitive defense to detect a breach that is about to happen or has just happened, and each detected breach should have an automatic response that is pre-strategized for immediate and maximum impact. Your security plan should have very detailed reactions for each detection tool in place.
Example: In reference to example threat #1 in our simulation, you could insert surrogate students into the instruction and testing process, and have a plan in place for those students to immediately notify you if an instructor provides inappropriate assistance. Your reaction plan could then be to immediately replace the instructor, launch an investigation, and fire the instructor if the allegation is proven. The important result is that you have stopped the threat in its tracks and minimized any damage.
It is imperative that all three solution types—prevention, deterrence, and detection + reaction—should be used in harmony against every threat. When in doubt, remember that experienced individuals on our team can help you craft solutions specific to your program's risks when needed.
Periodically evaluate security solutions and revise them as necessary. You should have several indicators (your detection system) in place to tell you how well your solutions are working. These include media reports (or lack of), Web Patrol® reports, data forensics reports, frequency of tips, test administration logs, and others. Other types of feedback can be solicited from time to time from program employees and stakeholders. All of these steps keep you aware of the health of your program. Based on this feedback, solutions can be kept as they are, fine-tuned, changed considerably, or replaced entirely.
Repeat Steps 1-3 at least once per year. New threats will continue to emerge, and you will want to be ready for them before they have the chance to impact your tests or the validity of those test results. Part of the reason that security problems are rampant today in high-stakes testing programs is that a reasonable, budgeted, planned security process is not in place. It's time to change that—and this 4-step process is the best way to do that.