CAVEON SECURITY INSIGHTS BLOG

The World's Only Test Security Blog

Pull up a chair among Caveon's experts in psychometrics, psychology, data science, test security, law, education, and oh-so-many other fields and join in the conversation about all things test security.

The Test Security Process: 1) Prevent, 2) Deter, 3) Detect + React

Posted by Caveon

updated over a week ago

Introduction

In a previous article, we outlined all possible threats to your testing program and showed you how to determine which specific test security risks are the most significant for your individual program. The next step in this process, which this article will outline, is how you can develop a plan that focuses your priorities and resources on those risks particular to your program. Let’s get started.

How to Develop a Plan for Your Program's Specific Test Security Risks:

  1. Think In Terms of "Assets" vs. "Threats"
  2. Embrace the Protection Process
    1. Prevent Security Incidents
    2. Deter Cheating and Theft
    3. Detect + React to Security Threats
  3. Repeat

Think In Terms of "Assets" vs. "Threats"

Most exam programs don't think in terms of protecting an asset, but there is no better way to illustrate the purpose of test security. The most important asset for your testing program is your test scores. When trusted completely, these scores can be used for many valuable and important decisions (like whether a medical student is qualified to become a doctor or a student is prepared for the next grade level).

A secondary asset is the exams that are taken to produce those scores—it is important to make sure that those exams are not stolen or shared for the benefit of cheaters, which would negatively affect the assessment program’s most important asset.

Simply put, the key to a successful security plan for any testing program is to understand how to protect your assets from your threats—which are all laid out here.

Embrace the Protection Process

The second recommendation is to embrace the Protection Process, commonly referred to as “the Test Security Process.” Protecting your two most important assets, your test scores and your exams, requires three classes of behaviors. Each class of behavior exists to counter the effectiveness of known test security threats within each category. The Protection Process involves (1) prevention, (2) deterrence, and (3) detection + reaction, in that order. You can think of it like this:

Test Security Process Stool

In order to have a well-rounded, sturdy, and holistic test security program, it's important to develop strategies and plans that cover all three categories of the Protection Process: strategies that (1) prevent, (2) deter, and (3) detect + react to threats. It is inevitable that unfortunately, sooner or later, breaches will occur. But when they do, you can be fully prepared to execute strong tactics within all three of these foundations, enabling you to keep your most important asset—your test scores—valid. Let's dive into each how each “leg” of the Protection Process works together to support a truly successful assessment program.

 

1. Prevent Security Incidents

Prevention consists of actions a program can take to actually stop or reduce the risks associated with the threats before they even get off the ground. (An added benefit—while not its primary purpose, preventative solutions can have a significant deterrent effect as well when the scope and effectiveness of those solutions are communicated broadly.) Preventative measures are versatile and can (and should) be implemented at every stage of the assessment process—you can even design preventative test security measures directly into your exam questions (with innovative self-protecting items like these). Preventative measures include, but are not limited to:

    • A test security plan that specifies the processes your organization will follow to prevent test theft. (You can learn more about how to create and execute a proper test security plan with these articles.)

    • A candidate agreement that lets test takers know what is/is not allowed in terms of test preparation materials, materials used during test administration, appropriate clothing to be worn, prohibited electronic devices, sharing test content, and the potential sanctions should any of these rules be broken.

    • Test development protocols with access rights to test content spelled out, confidentiality agreements signed by Subject Matter Experts (SMEs), and service-level agreements decided between the testing organization and the test administration provider.

    • A Security Incident Response Plan (SIRP) that specifies the actions that will be taken when a breach occurs. This includes those who will be involved in managing and mitigating the investigative process, what the outcomes will be to those who committed the breach, and how the investigation’s results will be communicated to stakeholders.

    • Secure Test Designs such as Computer Adaptive Testing (CAT) and Linear On-The-Fly Testing (LOFT).

    • Secure Item Designs such as the Discrete Option Multiple Choice™ (DOMC) item and the SmartItem™, which can prevent cheating and test theft from ever taking place. (With DOMC™, options are randomly presented one-at-a-time [think of a multiple-choice item, but more secure]. With the SmartItem, you have a self-protecting item that uses special technology during item development to appear differently to each test taker each time it is administered. Both of these innovations have been proven to prevent and deter many forms of test fraud and cheating, provide more accurate and useful test scores, and save time and money by reducing content exposure. You can learn more about both of these secure item designs here.)

 

2. Deter Cheating and Theft

The preventative strategies that your organization defines will flow nicely into the next set of strategies—deterrence. You can think of deterrence as a communication plan meant to discourage and inhibit test takers from cheating or committing other forms of test fraud. Deterrence is often just letting would-be test cheats and thieves know that there are consequences to their actions.

A good analogy for thinking about deterrence are the surveillance cameras used in retail stores. Through signage, businesses communicate their security measures that can catch shoplifters in the act, deterring would-be thieves from even attempting to steal merchandise. The same is true of communicating test security policies to test takers; by communicating the consequences of being caught cheating, you can deter individuals from ever attempting to commit fraud in the first place. (And, a plus side—when you utilize detection measures like data forensics and Web Patrol®, you can communicate that you have sophisticated test security measures in place that will catch cheating even after it has occurred! There’s no better deterrent than that.)

Another notable outcome of deterrence is that it lets individuals know what is and is not acceptable. Many examinees have a sincere desire to honestly take their exams. Having clear guidelines and communicating what is allowed before, during, and after the exam can help honest test takers avoid any “gray” areas when preparing to take their test, providing assistance to other test takers, or providing information about the test that shouldn’t be shared. (You can learn more about various deterrent measures—including proctoring—with these curated articles.)

Other examples of deterrence include (but are not limited to):

    • Communicating to test takers that:
      • Only sanctioned or authorized test preparation materials from the testing organization are allowed.
      • Proctors are present, and have been trained how to identify cheating.
      • Social media platforms and chat rooms are being monitored before, during, and after testing windows.
      • Special statistics can uncover dishonest test-taking practices after they have occurred—including examinees utilizing proxy-test takers and item pre-knowledge.

    • Communicating to subject matter experts, teachers, and other groups that:
      • Testing materials must be securely stored and accounted for, and checks and balances are in place to ensure they are.
      • Unauthorized access to secure test content and/or soliciting, receiving, distributing, or using secure test content is in violation of the contract, and action is always taken when those rules are not strictly followed.
      • Assisting test takers, pre-filling answers, or manipulating assessment results is in violation of the rules, and measures are in place that enable you to find those occurrences.

3. Detect + Quickly React to Security Breaches

At this point, you’ve done your best to prevent test theft and cheating from occurring. However, how will you know when a breach has occurred? This is where the “detection + reaction” process comes in.

“But that’s two steps,” you say. While it can be viewed that way, it’s vital to mesh detection and reaction strategies into one smooth and seamless process. For example, if you uncover the very beginning of a test security breach (detect), that incident is associated with an immediate, specific, and almost automatic pre-determined set of responses (react). Those “automatic” responses have been strategically pre-planned and formulated to stop that specific type of breach from harming your test scores, and they have been custom-developed to quickly and efficiently mitigate any damages that have already occurred.

For example, let’s say your web monitoring team notifies you of a braindump site selling your live test questions. You’ve detected a test security incident. Luckily—since you’ve already developed an effective security incident response plan for this exact situation—your team can immediately respond without wasting any time. So, without a second thought, you immediately send a copyright infringement notice to the braindump site, remove and replace the affected items, and conduct data forensics to uncover exactly which test takers utilized the braindump content. With all of these pre-planned measures in place, you’ve 1) detected a security violation, 2) responded quickly, mitigating damage, 3) empowered your program to carry out ramifications, if desired, and 4) enabled you to evaluate the results of your detection + reaction process and determine whether changes should be made. But above all, you’ve protected your most important asset—the validity of your exam scores.

As with preventative measures, detection + reaction strategies can have a significant deterrent effect as well, but only if the scope and effectiveness of those solutions are broadly communicated (learn how you can do that in this article).

Another benefit of the detection + reaction process is its direct impact on the answers on which test security’s most important questions are framed: Are there any test takers that have gained an unfair advantage on my exam? What impact does this breach have on my exam’s performance? Can my test takers’ scores be trusted? Detection activities include, but are not limited to:

    • Web and media monitoring
    • Data forensics
    • Anonymous tip lines
    • Proctoring
    • Search engine notifications (Did you know you can get alerts when relevant content is published?)

Of course, the actual reaction to the breach that your detection methods uncovered will vary greatly depending on the specific incident, so we won’t go into that here. But if you’re looking to get started on or fortify your incident response plan, these articles and this workbook are a great place to start.

Coming Full Circle

Now you have come full circle. You have:

  1. Gained an understanding of all the threats that can pose a risk to your testing program
  2. Determined the threats that pose the greatest risk to your specific program
  3. Prevented threats through thoughtful planning and multifaceted security measures at every stage of the assessment process
  4. Deterred cheating and test theft by communicating consequences to your test takers and subject matter experts, and assuring them that you have expansive measures in place that can ascertain test fraud
  5. Detected + Reacted to any breaches that your preventative and deterrent measures were not able to circumvent.

The last step is to repeat this process. Test security prevention is never fool-proof. Just when you think you’ve covered all your bases, someone will find a new way to cheat or steal your items. But don’t give up; keep fighting. Test security just needs a little love and a solid plan. By thinking constructively and working diligently, you can protect your two most important assets—your test scores and the exams that produce those scores—from the many threats that are present. Remember that each “leg” of the Protection Process—prevention, deterrence, and detection + reaction—must work together to support and secure your assessments and keep your exam results fair and valid. When usable, the results that your test produces can be trusted for valuable and important decisions.

About Caveon

For more than 18 years, Caveon Test Security has driven the discussion and practice of exam security in the testing industry. Today, as the recognized leader in the field, we have expanded our offerings to encompass innovative solutions and technologies that provide comprehensive protection: Solutions designed to detect, deter, and even prevent test fraud.

Topics from this blog: Test Security Consulting Data Forensics Exam Development Investigating Security Incidents K-12 Education Web Monitoring Test Security Basics Monitoring Test Administration DOMC™ Detection Measures Deterrence Measures Prevention Measures Certification Test Security Plan Higher Education Online Exams SmartItem™ Security Incident Response Plan Medical Licensure