The World's Only Test Security Blog
Pull up a chair among Caveon's experts in psychometrics, psychology, data science, test security, law, education, and oh-so-many other fields and join in the conversation about all things test security.
Posted by David Foster, Ph.D.
updated over a week ago
Security problems associated with traditional test administration models and advances in testing technology have encouraged the use of online proctoring, the rapidly growing alternative for monitoring online high-stakes tests. A new corps of trained online proctors use a variety of methods to deter, detect, and deal with cheating as it occurs (read this article about the effectiveness of proctoring in preventative, detective, and deterrent endeavors or this white paper on proctoring measures to learn more). Included in the methods that online proctors use is video from capable webcams, control over the test session, chat-based communication, automated monitoring, and unique authentication software methods.
Research indicates that online proctoring methods, when properly applied, can be as or more effective than traditional proctoring occurring in testing centers. Besides curbing security problems, the online approach provides students and other test takers the convenience of taking tests in their homes, schools, or workplaces. It also reduces costs associated with testing—such as testing fees, travel costs, parking passes, and other related expenses.
In addition to online proctoring’s effectiveness out of the gate, one of the most exciting aspects of remote proctoring is the use of technologies that will improve rapidly and increase proctoring's effectiveness over time. Technology such as artificial intelligence and machine learning for monitoring candidates with facial recognition; the automatic detection of hidden devices; the use of data forensics; or test designs that reduce overall item exposure, such as AIG or SmartItem technology. You can read this article about proctoring technology and techniques of the future to learn more about the technological advancements currently available.
Unfortunately, not all online proctoring services are equivalent. While some services are suitable for high-stakes testing, some others actually increase the risk of cheating. Examples include:
1. Services that allow proctors to view the test taker’s screen during an exam. This is a serious security and privacy risk. Proctors who have this ability could, in theory, view a test taker’s screen, see the question presented, and identify the answer provided by the test taker. There is simply too much opportunity for both collusion and harvesting of test content.
2. Another example is related to laptop cameras, which provide, at best, a head-and-shoulders view of the examinee. These cameras, while less expensive and easier to use, don’t provide the proctor with a sufficient view of the testing environment during the exam to detect most attempts at cheating. Additionally, these cameras cannot spot hidden cameras or other testing contraband (view this white paper on proctoring hidden cameras [pg. 8-10] to see exactly how much proctors can detect using standard laptop cameras).
3. A third example is the privacy concerns and authentication failures of some services when authenticating students. Having examinees display a government-issued ID close to the webcam lens is not an effective authentication measure. Not only are there issues with privacy and ease in the falsification of government-issued IDs, but there are also difficulties in verifying IDs in the best of circumstances—let alone through a webcam lens. Such an authentication method would be easy to circumvent (see if you can tell if this ID is real or fake on page 8 of this white paper).
Above are just three examples of cheating and test theft that proctoring cannot adequately detect (and may even introduce). There are many other forms of cheating and theft that proctoring cannot detect or prevent as well—among these are examinees using pre-knowledge or hiring proxy test takers. You can learn more about these and other threats in this document outlining the security threats proctors can and cannot detect and this white paper listing the possible test security threats a program can face. Lastly, if you're currently in the process of selecting a proctoring vendor, view this article about the five security considerations you should keep in mind when hiring a proctoring service.
As in any market, there are entrants willing to cut corners without understanding or disclosing the security problems to an unwary testing program. It isn’t until later when widespread cheating comes to light that the testing program recognizes the flaws. Until online proctoring services understand the many possible security risks and create technology and procedures to protect the test and detect cheating, online proctoring will remain appropriate for only low-stakes exams.
For further reading about online proctoring, I recommend these resources:
A psychologist and psychometrician, David has spent 37 years in the measurement industry. During the past decade, amid rising concerns about fairness in testing, David has focused on changing the design of items and tests to eliminate the debilitating consequences of cheating and testwiseness. He graduated from Brigham Young University in 1977 with a Ph.D. in Experimental Psychology, and completed a Biopsychology post-doctoral fellowship at Florida State University. In 2003, David co-founded the industry’s first test security company, Caveon. Under David’s guidance, Caveon has created new security tools, analyses, and services to protect its clients’ exams. He has served on numerous boards and committees, including ATP, ANSI, and ITC. David also founded the Performance Testing Council in order to raise awareness of the principles required for quality skill measurement. He has authored numerous articles for industry publications and journals, and has presented extensively at industry conferences.
View all articlesFor more than 18 years, Caveon Test Security has driven the discussion and practice of exam security in the testing industry. Today, as the recognized leader in the field, we have expanded our offerings to encompass innovative solutions and technologies that provide comprehensive protection: Solutions designed to detect, deter, and even prevent test fraud.
Topics from this blog: Proctoring
Get expert knowledge delivered straight to your inbox, including exclusive access to industry publications and Caveon's subscriber-only resource, The Lockbox.
