In order to exponentially improve the security of your testing program, what are four things should you do to enhance your exam’s security?
Test security for exams: it's no small undertaking. With so much to consider, how do you determine which security endeavors will provide the highest return on investment for your time?
The good news is there are four key enhancements that, when applied, are proven to significantly enhance the security of your testing program. They are:
Each point is critical for the security-success of your testing program. Let's go more in-depth into each of these points.
In every industry where security is involved, there is a routine process that occurs when a threat is detected. The response is automatic and happens immediately and seamlessly—and the testing industry should be no different. One of the key things an organization must do is prepare a plan ahead of time.
This pre-planned reactive strategy is typically called a Security Incident Response Plan (SIRP) (SIRP), and it is part of the bigger Test Security Plan). Everything within this pre-written blueprint goes into depth about specific security incidents that are likely to occur and the rules that are in place for when they occur; if one thing happens, this is what is done about it.
For example, if a statistical anomaly passes a threshold, the response could be starting an investigation and issuing a re-take of the exam or canceling the score. If it’s serious enough, the response may be to ban the examinee from the program. Whatever it may be, the reaction that occurs in any type of detection system must be pre-planned; there won’t be any time—and there is certainly no reason—to meet as a committee and make a decision impromptu after the breach has occurred. It should an automatic process.
A few questions to think about for your program’s Incident Response Plan:
Follow each of these questions (and many more) with a well-thought-out reaction.
For additional help getting started on your Test Security Plan or Incident Response Plan, the ATP and ITC guidelines point out the different areas and sections that should be included in your security plan, and the Security Boot Camp Workbook Part 1: Preparedness provides actionable tips that help you directly apply these concepts to your program. Additionally, Episode 4 of Questionmark's podcast, "Unlocking the Potential of Assessments," hosted by John Kleeman, provides more information about the risks and threats your program might face throughout the testing cycle.
In all, plan ahead of time. Don’t put a committee together on the fly when a security breach is discovered, and never leave the important decisions up to the last minute.
This specific category is a passion of the Caveon family. Technology today is remarkable, and while cheaters and thieves use advanced technology to cheat on exams and steal and distribute exam content, testing organizations can use it to design tests that make it really hard—if not impossible—to cheat. Items and tests that protect themselves, such as the Discrete Option Multiple Choice™ (DOMC) Item and the SmartItem™, have changed the traditional way testing works. Tests still measure the same, but now they protect and prevent test fraud while they are being administered.
Other innovations like AIG make it possible to generate thousands of unique items and efficiently expand your item pool, and Computerized Adaptive Testing (CAT) adapts to the examinee’s ability level.
In all, testing programs large and small can implement new testing technology, regardless of budget amount. Look into different item types that could benefit your program and investigate platforms that can help with test design and give your program the best chance at preventing test fraud straight out the gate. With it now being possible to build security straight into the test and prevent security problems, there is no reason any test should be administered unprotected.
When Caveon first started in 2003, high-stakes testing programs would employ anywhere from zero to sometimes two security experts. Today, there are high-stakes testing programs who employ dozens of security experts. Regardless of the number of people your program has, you should always have at least one person on staff who’s your dedicated security person.
For those just getting started, there’s a plethora of material available for training people specifically on the intricacies of test security for exams (such as the TILSA Test Security Guidebook co-authored by Caveon’s Dr. John Fremer, the ITC Guidelines, and Caveon Insights). Regardless of your program’s size or age, having someone on staff dedicated to implementing your test security plan and managing your testing program’s security-related efforts is a crucial effort that will set you apart. It is the foundation for a successful testing plan and a successful testing organization.
Testing organizations often falsely assume that they don’t need to think about test security for exams until a breach occurs. This is simply not true. In today’s technology-enhanced world, it is best to assume that your program will come across a security breach at some point.
With this in mind, many programs have opted to hire a testing vendor. What oftentimes then happens is the testing organization assumes most or all of their security-related responsibilities can be ignored and that their vendor—their test administration vendor in particular—will take care of everything security-related. But this is not the recipe for success! The ultimate responsibility for a security breach or for a badly designed and administered exam lies with the testing program itself.
Think of your relationship with your vendor as a partnership whereby you and the vendor are working together to prevent, detect, and deter test fraud. Take responsibility for security and really get the best out of your vendor by delegating tasks for them to do that will enhance your security.
Examples of this could be siphoning off your web monitoring efforts to a team dedicated solely to scouring the dark holes of the internet for leaked test content, or utilizing advanced item types or test designs such as DOMC™ or the SmartItem™ on your exam. One of Caveon’s most utilized services is our data forensics service, where our trained psychometricians evaluate testing data to uncover otherwise invisible cheating efforts.
If you implement each of these four things at a high level—formulate a test security plan ahead of time, design items and tests that protect themselves, have a security expert on-hand, and take full responsibility for security—your program will have fewer security problems and be better prepared to handle any that come your way.