The World's Only Test Security Blog
Pull up a chair among Caveon's experts in psychometrics, psychology, data science, test security, law, education, and oh-so-many other fields and join in the conversation about all things test security.
Posted by David Foster, Ph.D.
updated over a week ago
With higher education (and especially higher education exams) moving more toward the online sphere, it’s more important than ever to ensure the tests your institution gives are valid. The biggest threats to the validity of your test scores are cheating and theft; and when it comes to cheating and theft, prevention is always the best medicine. So, how can you prevent cheating in the online exams your college or university administers? Let's talk through some strategies.
The amount of cheating by college students has been researched extensively over the past three decades. This is due, in part, as colleges’ methods for administering exams have changed rapidly from paper-and-pencil exams, to computer workstations at college testing centers, to online testing using personal computers in any location. The unfortunate (and well-researched) truth is that cheating in higher education is rampant, with more than half of students admitting to cheating on university exams.
Additionally, the amount of cheating is not going down. Rather, research suggests that cheating is actually increasing with the transition to online exams, where students take tests in homes or in dorm rooms, sometimes even without proctors. The key takeaway here is that cheating in higher education, while probably getting worse, was already at a mind-boggling level.
From a test security perspective, higher education institutions have done very little to stop the rise of cheating. (Especially when compared to the test security efforts of K-12, certification, licensure, and other industries that administer high-stakes tests.) We can only hope that the current transition to online testing and online proctoring is the impetus to finally addressing the rampant cheating problem.
The good news is that cheating in higher education is relatively easy to address. And it can even be addressed by implementing a few very simple steps. I’ve divided my recommendations into short-term (0-6 months) and long-term (1-3 years) security efforts.
What makes us so confident that we can successfully manage student cheating? The methods of cheating used by most college students are simplistic. Most students simply take advantage of the poor test administration practices surrounding the creation and administration of college quizzes, midterms, and final exams. A few examples of poor testing practices that universities often employ include: creating sub-standard exams, administering exams through simplistic LMS testing software, poor or non-existent proctoring, using the same test questions over and over again, and inadequate security training for professors and TAs.
Poor testing practices like the ones listed above provide ample cheating opportunities for students. Often, higher education students view cheating as a “crime of opportunity.” However, by removing the opportunity instead of providing it, universities can do a lot to stop the rising tide of cheating. Most students will certainly not cheat if either the easy opportunity is taken away, if there’s a healthy chance of getting caught, and/or if the punishment for cheating is stiff enough.
The methods students use to cheat are not the same as they were a decade ago. With new technology and new ways of testing have come new ways to cheat. The best way to understand the ever-evolving methods for student cheating is to use a framework that will apply even as cheating evolves. The best framework is to consider cheating from the standpoint of the “threat” it poses. Caveon has produced a table of threats here. (Of note, this table covers all “test fraud” threats. Test fraud is a blanket term used to describe efforts to cheat and efforts to steal test questions.)
What is the difference between cheating and stealing test questions? A student that memorizes questions to share with their friend is not cheating, but stealing. The friend that uses the stolen information to get a better score on a test is cheating. Both are committing test fraud, but only the second is guilty of cheating. However, because stealing test questions directly leads to cheating, universities need to stop that as well.
Using the framework above, these are the main ways students can use to steal test questions and cheat on exams:
Using pre-knowledge of the test content
Having someone take the test for them
Using cheat sheets and braindump sites
Capturing test questions using a camera (for example, with a cell phone or a hidden camera)
Memorizing questions and recalling them at a later time
To save time and money, make sure to focus your efforts to stop cheating in higher education on these specific threats. You can learn more about how to determine the threats specific to your testing program in this article.
Before listing the test security solutions for college testing, it is vital we first discuss the fundamentals of test security. There are three categories of security solutions: those that Prevent, Deter, and Detect + React to security threats. This trio is often referred to as the Test Security Process.
Why is this important? If a university or college wants to successfully stop cheating, they need to combine test security measures from each of these three categories (after all, who wants to provide a diploma to any student who cheated their way on important exams?).
Test Security Methods that Prevent Cheating & Test Theft: Stop students from cheating on exams and/or stealing and sharing test questions.
Test Security Methods that Deter Cheating & Test Theft: Persuade students that it is not in their best interest to cheat on an exam. This is done by convincing students that cheating is wrong, that they are likely to get caught, and/or that the punishment is more than they are willing to risk.
Test Security Methods that Detect + React to Cheating & Test Theft: Catch those students who bypass your prevention and deterrence efforts and cheat on an exam, and then swiftly respond to the breach.
Let’s now look at the specific solutions higher education institutions can use to prevent, deter, and detect + react to cheating on exams. I assure you, these solutions are straightforward. More than that, they have a high likelihood of success. Using these solutions is easier, and it brings less risk (even in the short-term), than the current path universities are on; the path that actually encourages cheating and test theft through uninformed testing and test security practices.
I’ve divided my recommendations into short-term (0-6 months) and long-term (1-3 years) security efforts.
Proctors are not great at stopping cheating (read that again). The over-reliance on proctors is blocking most colleges and universities from implementing far better and far more effective security solutions that protect their assessments. In the end, this over-reliance on proctors (especially when at the expense of better test security tactics that could be used) actually increases cheating rather than stops it. To learn more about the security bandwidth and limitations of proctors, view this infographic or this article, or read this white paper. Case studies show that once you begin utilizing other solutions, rather than relying 100% on proctors, test security increases. You will have taken the first—and most important—step towards effectively stopping cheating, and the validity of your institution’s test scores will be better protected.
Until you have the long-term security solutions in place, never make the decision to give important tests without proctoring at all. I know that proctors are expensive, and that there are debates over student privacy. However, allowing students to take exams without even the minimum security provided by proctors is handing students opportunities to cheat. Instead, find a proctoring solution that adheres to your universities policies and best practices. Then work on implementing the other security solutions from this list that will better protect your assessments. At some point, proctors may not be needed for your assessments at all.
This is the easiest solution to implement, and you are likely already doing this to some extent. A common example of a deterrent communication includes requiring students to sign an honor code for each exam. Brainstorm even more methods for how students can be convinced not to cheat or steal test content, and then go ahead and use every idea you come up with (even the bad ones). Track how these communication strategies work and determine which methods you’ll keep long term. For example, consider a university-wide push to inform about ways cheating happens, why it is hurting students’ chances of success, the diminished value of a graduation credential, etc. In that campaign, explain the new rules, the new systems in place, and a new university commitment to stamping out test fraud. Publicize widely the consequences for cheating, including any examples of students who were caught and punished. The goal of deterrence communications is persuasion. You can learn more about deterrence communications in this article.j
Professors should stop using test questions that were given last term (or last year). It’s hard, I know. Professors are already overworked and busy. However, school administration should provide the resources necessary that allow professors and TAs to write fresh questions before each exam. (Even if those new ones only have slight modifications.) Make the students believe that they cannot rely on what they or others had seen previously, that they cannot simply ask a friend who took the course the previous year to tell them what will be on the test. (On a positive note, see the long-term solution #2 to learn about a new innovation in testing that allows you to write test questions that never need to be changed or refreshed, and that are never repeated.)
To keep students from taking tests for others (what is known as proxy testing), upgrade your authentication system. Put stronger identification steps into place (for example, require two ID documents instead of one), or add a biometric authentication measure. Use biometrics that are one-to-one match comparisons. Often, the student voluntarily provides the biometric at the beginning of the year or semester, and then they simply need to reproduce it prior to taking the test. An example of this form of biometric is a keystroke pattern system. I do recognize that there are privacy concerns surrounding biometric data (such as requiring fingerprints or facial recognition), but it is entirely possible to only use those biometrics that don’t require specialized hardware and that adhere to university privacy standards.
Abandon the simplistic test and question designs supported by paper testing or your current LMS. Not only do these antiquated designs result in poor measurement, but they also put your institution at substantial risk for cheating and test theft. Make the decision to transition to professional testing systems that incorporate security directly into the designs of questions and tests. The best system today is Caveon’s Scorpion. It’s the only system in the world that was built by test security experts for the sole purpose of secure testing and obtaining valid exam results.
The SmartItem is a complex item that uses special technology when being written so that it measures an entire learning objective in one test question. On a test, the SmartItem renders a unique question to each student (making it pointless to memorize or steal test content and share it with others.) Additionally, when a SmartItem is written for a course, that SmartItem can be re-used semester after semester, year after year, without loss of quality and without an increased risk of cheating. (Learn more about the foundations of SmartItem technology in this infographic and this booklet, and see an exam SmartItem written for a college psychology course here.)
An easy fix, you can instantly boost the security of your tests by using the DOMC™ item type instead of regular multiple-choice items. The DOMC item type presents multiple-choice options one at a time rather than simultaneously, which makes it very difficult for students to steal and pass on useful information. Learn more about the DOMC item type in this case study and this white paper.
Monitor the web to uncover where students are discussing your exams and sharing your institution’s test questions. This includes forums, chat rooms, and social media sites. Consistent web monitoring will also locate companies that are offering to take tests for your students, or that provide plagiarized term papers and even fake diplomas. There are services that specialize in uncovering student cheating and can conduct monitoring for you, enabling you to focus on quality tests and test designs.
Conduct routine data forensics analyses of student testing data. Data forensics enables you to identify anomalous patterns in test results that indicate cheating. Read this case study to see how truly efficient data forensics is at uncovering cheating in an educational setting. Conducting data forensics is truly one of the best ways you can detect even small instances of cheating.
Sign up for an audit by test security professionals to determine the strengths and weaknesses in the security efforts by your university as a whole. Repeat the audit every 3-5 years. You should use this audit as an opportunity to prepare a test security handbook for your university that is approved by stakeholders and lists your university’s policies and procedures around test security.
Over ten years ago, Cluskey, Ehlen, and Raiborn (2011) provided eight essential test security recommendations to higher education institutions. As I mentioned earlier, cheating and test theft efforts update and get more sophisticated with time, especially as new technology emerges and changes the levelness of the playing field. With that in mind, I would like to briefly review those recommendations in light of the many changes this past decade has brought. To make this easier, I’ve assigned a letter grade to each of their suggestions and provided an explanation for its current security effectiveness. I applaud Cluskey, Ehlen, and Raiborn for making an effort to improve the security of online tests. Their suggestions were appropriate for the time, and a few remain relevant even today.
Grade: D
Reason: Giving an online test simultaneously to students across the world, or even just at a university’s test center, is not feasible. One of the main benefits of online tests is that they are convenient for test takers. A testing window can be established, but it shouldn’t eliminate the benefits of flexible testing. Within a given testing window, 24/7 testing is the standard. Requiring all students to take the test at the same time does not eliminate many of the most effective forms of cheating (such as using pre-knowledge, proxy testing, the problems that stem from re-using test content each semester, etc.). There are better methods available that stop cheating, but that still allow a flexible testing window.
Grade: D
Reason: This suggestion is similar to the first, and the same critique applies to this as well.
Grade: B
Reason: This recommendation remains as solid as it was ten years ago. Such randomization will make creating “answer keys” that are easy to stuff in your sock while taking a final exam more difficult to do. It also makes it impossible to benefit from looking at the test of the person seated next to you taking the same test as you. However, the reason I did not give this an “A” is because cheating by looking at the person next to you or by using a cheat sheet are not the most damaging or most common forms of cheating. In today’s online testing environment, stealing every question on a test with hidden cameras, and then sharing that test content broadly on social media and braindump websites is a much more serious problem. Then other test takers utilize their pre-knowledge to pass the exam. Unfortunately, randomization does nothing to prevent this.
Grade: A
Reason: This is an excellent recommendation. Being able to return to previously seen questions is a leftover artifact of giving tests on paper. Additionally, it has doubtful value, it takes up important testing time, and it enables the theft of test content and cheating. All universities should eliminate the ability to go back to previous test questions on their online exams.
Grade: C
Reason: The intent here is to restrict the test’s time limit in order to avoid giving examinees the time they need to cheat. While the intent is noble, there is no research to suggest that such a restriction will have any impact on cheaters. On the flip side, test takers of all abilities take exams at different paces. The different rate at which examinees answer questions can be based on several factors, including language experience, personality, accommodations, etc. A time limit would only serve to limit those honest students. As a much better alternative, however, a program can conduct data forensics analyses that monitor the response times of test takers and can identify anomalous timing patterns that uncover cheating. Actions, such as canceling a test score, can then be taken if those patterns are sufficiently unusual. This is a far more effective way to determine who cheated, and it doesn’t limit honest students.
Grade: D
Reason: This recommendation is based on an outdated factor. In the past, certain online LMS testing systems did not keep track of student progress through the exam (so when an examinee ran into technical difficulties, the test could not be restarted at the appropriate point). Today however, especially with professionally designed testing systems (please note that LMSs are not professionally designed testing systems), such a recommendation is unneeded. If a test is disrupted, it can, with proctor permission and assistance, be continued.
Grade: C
Reason: Several lockdown browser programs were available a decade ago, and even more are on the market today. These lockdown browsers are advertised to prevent certain behaviors during the test. For example, the program might disable the hotkey combination used to take a screenshot, or it could disable a test taker from opening a new internet tab or accessing their personal files. Such lockdown programs have drawbacks, however. They are fairly easy to circumnavigate, and they are invasive in that they require installation on a student’s computer or device prior to testing. Today, the jury is still out on the usefulness or legality of using such programs.
Grade: B
Reason: This recommendation is laudable in that it recommends regularly refreshing the question pool from which exam questions are drawn. This is done in an attempt to counter the tendency and ease of just using the same questions term after term. However, this suggestion does have a couple of weaknesses. First, the “one-third” value is arbitrary with no real science to support it. Second, it doesn’t go nearly far enough. Today’s technology makes it possible to generate “once-and-done” questions on the fly during exams that are only ever seen by a single student. For example, see Caveon’s SmartItem for one way this can be done.
I appreciate Cluskey, Ehlen, and Raiborn and their efforts to stem the tide of student cheating a decade ago. Even today, several of their recommendations remain sound advice for online universities. However, with advancements in technology and the ever-evolving deviousness of cheaters, an updated set of solutions is necessary.
I now want to address a common idea that has been advocated recently, that higher education institutions should allow open-book testing in order to prevent cheating. However, there is no beating around the bush; allowing open-book testing as a way to prevent cheating is a terrible idea.
Don’t get me wrong, I do not think that “open-book” exams should not exist. The “open book” approach is valuable when one of the purposes of the exam is to require students to locate arguments and ideas quickly using reference materials and use them in a timely manner. A test can and should be used to measure that fairly complex skill. It is a skill that is becoming more relevant in this digital information age.
With that said, “open-book” exams should not be used with the intent to prevent cheating. It does not work to stop cheating—and in many cases, it might even increase it. Let me explain.
A test is simply a sample of questions from all possible questions that could be asked. Consider this example: a US History textbook contains the basic information for thousands of questions, but the professor will only write 50 to include on the final exam. A student’s score on the exam is meant to provide an estimate of the student’s knowledge of all the possible questions pertaining to the knowledge in the textbook. The exam score is not meant to provide an estimate of the student’s knowledge of the answers to the 50 specific questions chosen by the professor. With that in mind, the method of allowing open-book testing will work if (and only if) students do not know what questions will appear on the exam.
However, if the questions are ever re-used, or if there is ample time to share responses between students, this type of exam sets students up for cheating with ease. The open-book approach is a royal road to getting a high score by cheating, and professors are often unwittingly complicit. Additionally (and from a learning standpoint), an open-book exam allows each student to simply discover the answers to the small number of questions, without needing to know the rest of the course content. The students will ignore important parts of the textbook and lectures, and they will still have the ability to do well on the 50 questions.
Now, if each test for each student is unique, then an open book test would be a fine testing method. Using the example of the 50 questions above, an effective (and secure) form of testing would be if each student randomly received their 50 test questions out of the thousands of possible questions for the exam. With today’s technology (like AIG and the SmartItem), such a test design is possible. Not only would this be an ideal test of a person’s understanding of the important concepts for the course, but cheating would be rendered almost impossible.
More than half of students admit to cheating in online college exams. However, addressing cheating in higher education is an achievable goal. Colleges and universities can stop cheating in online exams in the following ways (short term):
No longer relying on proctors
Utilizing proctors in conjunction with other security methods
Implementing deterrent communication strategies
Refraining from reusing test questions semester-by-semester
Using a robust authentication system
Moreover, colleges and universities can stop cheating in online exams by preventing, deterring, and detecting/reacting to cheating and test theft in the following ways (long term):
Incorporating security directly into their test and item designs
Utilizing SmartItem technology
Converting all multiple-choice items to DOMC items
Conducting web monitoring
Using data forensics analyses
Regularly auditing their test security policies and procedures
Test security best practices change with time. It is important for higher education institutions to apply the above principles, while also adapting with changing threats, in order to best prevent cheating and effectively protect the validity of their online tests.
A psychologist and psychometrician, David has spent 37 years in the measurement industry. During the past decade, amid rising concerns about fairness in testing, David has focused on changing the design of items and tests to eliminate the debilitating consequences of cheating and testwiseness. He graduated from Brigham Young University in 1977 with a Ph.D. in Experimental Psychology, and completed a Biopsychology post-doctoral fellowship at Florida State University. In 2003, David co-founded the industry’s first test security company, Caveon. Under David’s guidance, Caveon has created new security tools, analyses, and services to protect its clients’ exams. He has served on numerous boards and committees, including ATP, ANSI, and ITC. David also founded the Performance Testing Council in order to raise awareness of the principles required for quality skill measurement. He has authored numerous articles for industry publications and journals, and has presented extensively at industry conferences.
View all articlesFor more than 18 years, Caveon Test Security has driven the discussion and practice of exam security in the testing industry. Today, as the recognized leader in the field, we have expanded our offerings to encompass innovative solutions and technologies that provide comprehensive protection: Solutions designed to detect, deter, and even prevent test fraud.
Topics from this blog: Braindumps Test Security Consulting Exam Development Investigating Security Incidents Web Monitoring Test Security Basics Monitoring Test Administration Proctoring DOMC™ Detection Measures Deterrence Measures Prevention Measures Test Security Plan Higher Education Online Exams Automated Item Generation (AIG) SmartItem™
Get expert knowledge delivered straight to your inbox, including exclusive access to industry publications and Caveon's subscriber-only resource, The Lockbox.
LEARN ABOUT A BETTER WAY TO ADMINISTER HIGHER-EDUCATION EXAMS
