The World's Only Test Security Blog
Pull up a chair among Caveon's experts in psychometrics, psychology, data science, test security, law, education, and oh-so-many other fields and join in the conversation about all things test security.
Posted by David Foster, Ph.D.
updated over a week ago
Proctoring has been used for over 1,500 years, yet our understanding of its impact is cursory. A proctor's effectiveness in addressing test security threats varies widely depending on other security factors. You may be surprised to discover where proctors excel and where you may need extra support.
Understanding how proctors affect the quality of test security efforts during test administration is not as straightforward as it may seem. Given that testing programs have been using proctors to monitor high-stakes exams for more than 1,500 years (since the beginning of the civil service examinations of Imperial China) and that proctors are considered by many to be the most critical security component of test administrations, it is shocking how little we know about what proctors really do with regards to test security.
In the past, proctors (sometimes referred to as test administrators) have played significant roles in cheating scandals. For example, in the widely-reported "Varsity Blues" college admissions scandal of 2019, proctors either participated directly in helping students cheat, or were bribed to turn a blind eye. At least one of the proctors involved has been indicted. In contrast to the implications based on the scandal, I have no doubt that the vast majority of proctors who monitor tests in-person or online fulfill their responsibilities with honesty and dedication. Proctors still play an important role in the security of test administration.
That said, it must be acknowledged that there are inherent risks involved with using proctors that are generally low-paid, or even volunteers. These individuals are usually proctoring on a part-time basis and fulfilling other administrative responsibilities beyond test security. They may not be well-trained in all the skills necessary for the job, and are rarely chosen specifically for their vigilance, commitment to the cause, or skillset in the tricky task of dealing with belligerent test-takers who are caught cheating. For those of us in the testing field, this creates a complicated dilemma—by utilizing proctors for test security purposes, we are also introducing test security risks.
To wrap our minds around this dilemma, we must first analyze and better understand the test security benefits that proctors provide, whether online or in-person.
The security solutions that protect tests can be divided into three types: Those that prevent misconduct, those that detect it, and those that deter it. (Read more about these three solutions in this article.) To understand the test security benefits of proctoring, we should evaluate how proctoring contributes to each of these three categories. Here is how I would rank proctoring based on my own experience from 37 years in the testing industry:
From a purely technical view, proctors cannot prevent any test security threats when monitoring test-takers. This means that, regardless of what the proctor is currently doing in the testing room, the test-taker can consult cheat-sheets, look at a neighbor, or carry out any other form of cheating at any time—none of which can be prevented by proctoring staff. Thus, the proctor’s role is a relatively passive one, limited to scanning the room, looking for misbehavior, and attempting to discourage cheating by just being present. These limitations make proctors relatively ineffective at directly preventing cheating or theft.
Proctors are trained to consistently scan the room of test-takers in order to discover any infraction of the rules, including the use of cheat-sheets or cell phones, communicating with others, etc. If proctors discover cheating or harvesting, they are trained (hopefully) to follow a specific protocol to deal with the incident. In this sense, they serve an important detection purpose, and are well-suited to uncover certain types of fraud. However, proctors are not equipped to deal with many of the most harmful forms of cheating, such as detecting the use of hidden high-tech devices, catching proxy test takers, or stopping individuals from using pre-knowledge to cheat on an exam. For this reason, they cannot be viewed as a comprehensive solution for detecting all cheating or theft during test administration.
Deterrence
Effectiveness Level: HIGH
A proctor’s most impactful test security benefit is in their ability to deter test-takers from engaging in fraudulent behavior. Simply by being present, whether it be online or in the room, proctors have a strong effect of convincing test-takers that it is too risky to attempt to cheat. Examinees know they are being watched, and must modify their behavior accordingly. This effect is just as strong—perhaps even stronger—when an examinee is taking an online test; they must assume that they are always being watched, and do not have the in-person advantage of waiting for the proctor to turn away or get distracted before cheating. Regardless of these nuances, it is clear that proctors create a strong deterrent effect.
The above analysis provides us with a fuller picture of how the role of proctoring plays into our test security strategies. It provides us with the valuable opportunity to reevaluate how we utilize proctoring, and to better understand how we can maximize its strengths and shore up its weaknesses.
It is obvious that proctoring is a valuable asset when it comes to deterring cheating and theft. Programs that utilize proctors (pretty much all programs!) should start by examining their current approach to proctoring, and then analyzing the changes they could institute in order to maximize proctoring’s deterrent benefits. This could include increasing the number of their proctors, implementing technology to assist them, improving their training, adding ways to make their job easier, and many others.
Testing programs should also consider taking a step back and looking at how proctoring fits into their wider test security framework. Since proctoring cannot be viewed as a one-and-done process for security during test administration, each testing program that utilizes proctors needs to honestly evaluate whether it is relying on proctors as their primary test security tactic for preventing and detecting test fraud. If so, this is a great opportunity to institute a myriad of other test security strategies that will not only fill the gaps that proctoring simply cannot address, but will also mitigate or lessen any of the risks presented by introducing proctors to a testing room.
For more than a millennium, high-stakes testing programs have relied on proctors to administer and protect our tests. Looking to the future, it is necessary that we closely evaluate just how proctors affect the security of our tests—where proctors improve test security, where they have no impact on test security, and where they make testing less secure. This article can hopefully serve as a starting point for us to better understand how we can deliberately employ proctors (in conjunction with other complementary test security measures) to ensure that our assessment programs are comprehensively preventing, detecting, and deterring test fraud.
A psychologist and psychometrician, David has spent 37 years in the measurement industry. During the past decade, amid rising concerns about fairness in testing, David has focused on changing the design of items and tests to eliminate the debilitating consequences of cheating and testwiseness. He graduated from Brigham Young University in 1977 with a Ph.D. in Experimental Psychology, and completed a Biopsychology post-doctoral fellowship at Florida State University. In 2003, David co-founded the industry’s first test security company, Caveon. Under David’s guidance, Caveon has created new security tools, analyses, and services to protect its clients’ exams. He has served on numerous boards and committees, including ATP, ANSI, and ITC. David also founded the Performance Testing Council in order to raise awareness of the principles required for quality skill measurement. He has authored numerous articles for industry publications and journals, and has presented extensively at industry conferences.
View all articlesFor more than 18 years, Caveon Test Security has driven the discussion and practice of exam security in the testing industry. Today, as the recognized leader in the field, we have expanded our offerings to encompass innovative solutions and technologies that provide comprehensive protection: Solutions designed to detect, deter, and even prevent test fraud.
Topics from this blog: Proctoring Deterrence Measures
Get expert knowledge delivered straight to your inbox, including exclusive access to industry publications and Caveon's subscriber-only resource, The Lockbox.
Learn more about proctoring in our white paper:
