CAVEON SECURITY INSIGHTS BLOG

The World's Only Test Security Blog

Pull up a chair among Caveon's experts in psychometrics, psychology, data science, test security, law, education, and oh-so-many other fields and join in the conversation about all things test security.

What Is Examinee Pre-Knowledge (and How Do You Stop It?)

Posted by Alison Foster

updated over a week ago

What is Pre-knowledge:

Examinee Pre-knowledge is when a test taker knows the questions or answers on a test before taking it. Stated another way, the term “pre-knowledge” is used to describe any information gained about the content of an exam prior to testing. It is a common and effective method of cheating.

Examples of pre-knowledge include (but certainly aren’t limited to):

  • Purchasing the answers to a test from a braindump website and memorizing them before taking the exam
  • Getting the answers to exam questions from a friend who has already taken it
  • Finding live exam content on social media or other social forums before taking the exam

You can learn more in this short video about pre-knowledge.

Play the Prior-Knowledge Video

How Does Pre-Knowledge Work?

Pre-knowledge as a form of cheating is a three-step process: First, test items (meaning the test questions and/or answers) must be stolen. Second, the stolen content is made available. Third, a prospective test taker accesses the stolen exam items, then memorizes them. The test taker then uses the memorized exam content to get a better score on the test.

  1. Theft of Test Content: In order for someone to cheat using prior-knowledge, live test content must first be stolen. It can be acquired by individuals using memorization, screen recording software, hidden cameras (the kind of cameras that can be concealed in a button, the frame of eyeglasses, a lapel pin, or simply appear to be part of the fabric of a shirt), etc. Stealing test content is difficult to detect, and it can be done by single novice individuals or by highly-professional groups. Please note that it is not necessary for all the questions and answers to be stolen (or even a high percentage of them) for examinee pre-knowledge to be useful.

  2. Stolen Test Content Made Available: After the test content is stolen, it is then distributed and made available to those who have yet to take the exam. Test content can be shared informally (or even unwittingly) via word-of-mouth, on social media, in chatrooms or forums, or sold for profit on braindump sites. There are dozens (if not hundreds or thousands) of ways test questions and answers can be distributed.

  3. Cheating with Pre-Knowledge: Test takers looking to illicitly increase their exam scores can track down stolen test content, memorize it, and use this newfound knowledge to unfairly get a better score on the exam.

Why should you care about pre-knowledge?

Pre-knowledge = fake knowledge.

Knowing all the answers to a test because you memorized an answer key is not the same as passing the test because of hard work and proper studying. Memorizing is not the same as true understanding.

Item pre-knowledge lets test takers fake their understanding and get a passing score that they did not deserve. As a result, you have unqualified individuals getting high scores on tests and passing their exams. This is often at the expense of honest, well-prepared test takers, and it is at the expense of society at large.

Devastating Results

The results of pre-knowledge can be devastating. Consider these real-life scenarios:

  • Scenario #1—An Undeserving Scholarship: A student memorizes test questions for a standardized test being discussed on an online braindump. Knowing the answers to these questions allowed this student to get a better score than was expected, and as a result, they qualified for a merit scholarship they would not ordinarily have received. In the end, an individual who honestly earned their score did not receive the scholarship.

  • Scenario #2—An Unsafe Licensure: An aspiring contractor is preparing to take the contractor licensure exam for their state. They are new to the construction industry and ran out of time to properly prepare for their exam prior to their testing date. In a panic, they found a website that promised a passing score for the low cost of $99. They purchased the exam key, memorized it, and earned a passing score—despite not being fully qualified or adequately understanding construction safety considerations. This contractor is now working alongside honest, qualified individuals in their city. Unfortunately, the contractor’s lack of knowledge is leading to sub-par work being done on people’s homes. In addition, it is contributing to an increased negative reputation among all contractors, and the value of the license, on the whole, is being called into question.

These scenarios have happened. Consider on a broader scale what can happen if society can no longer trust that its doctors, nurses, drivers, or teachers have earned their licenses honestly? (We’re wondering if that’s how this surgeon made it to the operating table.)

An Unfair Advantage

Allowing pre-knowledge to go un-checked, un-caught, and un-punished is unfair to honest test takers everywhere. Those individuals who worked so hard to achieve their understanding are directly and unfairly impacted by those who cheated. Whether losing opportunities (as we saw in Scenario #1), seeing their qualifications devalued (Scenario #2), or myriad other consequences, it is honest test takers who end up paying the price for another’s use of pre-knowledge.

Item Pre-Knowledge Is a Thriving Business

It is important, at this point, to emphasize that providing pre-knowledge is big business with a thriving marketplace of buyers and sellers. There are hundreds (if not thousands, internationally) of websites (known in the industry as “braindump sites”) that specialize in disclosing actual exam content, and new braindump sites are being created daily.

There is a vast array of affiliated blogs, filesharing sites, social media posts, videos, and other resources that direct examinees back to braindump content. With millions of dollars at stake, there will be people intent on profiting off of your live test content—and they'll be both creative and tenacious when it comes to building their businesses. And, in addition to those who are selling test content for monetary gain, there is an entire ecosystem of organic sharing to contend with: individual social media posts, online forums, apps, and word-of-mouth sharing between friends. All of these methods are used to share test content.

However, the intent for utilizing item pre-knowledge isn’t always nefarious. There are plenty of instances when individuals do not know that what they are doing is wrong. For instance, an examinee might post a screenshot of a test question on their Twitter account, only to realize later that individuals could use their post to cheat. Or they might share what exam questions they answered just to be helpful to a friend. Without thinking of the wider consequences, it can be easy for individuals to participate in knowledge sharing without the intent to cheat.

Regardless of the motivation, the vast array of methods for sharing exam content are only growing.

Is Proctoring Effective at Detecting Pre-Knowledge?

The only way to effectively stop pre-knowledge is through the four-step process (below). But before we dive into that, there is one point we feel is vital to get across: Proctors (no matter how diligent) are not effective at stopping or detecting pre-knowledge. Consider these questions with their respective resources:

  • Can a proctor (whether remote or in-person) tell if a test taker is wearing a hidden camera and stealing the test questions? The answer is a definite "No." I invite you to read this white paper or watch this presentation to see for yourself.

  • Can a proctor (whether remote or in-person) tell when a test taker has downloaded answers off of the internet, memorized them, and is using this pre-knowledge to cheat on an exam? I assure you, they cannot.

Proctors (both remote and in-person) cannot detect individuals who are cheating by using pre-knowledge. No proctor, however diligent, can tell the difference between an honest test taker and one who purchased and memorized the test questions.

So, if your program is utilizing proctors as your primary method of test security (which is the case for many testing programs), take this moment to consider the potential hole in your security. If proctoring is your primary method of test security, you are still vulnerable to pre-knowledge, one of the most effective forms of cheating.

How Do You Stop Pre-Knowledge?

The only way to effectively stop pre-knowledge is through a four-step process: utilizing secure test and item designs, creating a large (if not infinite) item bank, monitoring the web, and analyzing your test results.

1. Utilize Secure Test and Item Designs

Item design is how an item is created and delivered. For example, multiple-choice and matching are both straightforward examples of item design.

The design of your items and tests is critical in defending them from the effects of theft and pre-knowledge. Therefore, whenever possible, convert your multiple-choice items into more secure options (such as DOMC™ items) or embrace adaptive test designs (like computerized adaptive tests).

Most importantly, consider utilizing SmartItem™ technology. The SmartItem is a simple but significant innovation. It uses special technology during development and delivery so that the item renders differently each time it is given on a test. Translation—no two examinees see the same thing during testing, so using pre-knowledge to gain an advantage is useless.

When deployed, SmartItem technology and DOMC items make it so every test taker has a fair chance of displaying their knowledge and experience. They also have myriad other benefits.

2. Pull Test Content from a Large (If Not Infinite) Group of Items

A large enough item bank utilized properly has many benefits. It makes it so that in the likely event that live exam questions are stolen and exposed online, any exposed items are unlikely to be seen by others on the exam. Even if a test taker has memorized test questions and answers, the odds of it appearing on their test are so low that their effort to cheat is ineffective.

A large item pool also means that testing programs can replace items quickly in the event of a breach, limiting the damage from examinees utilizing pre-knowledge. In addition, an abundance of items also makes secure test designs (such as CATs, LOFTs, or even a plethora of forms) possible for all programs. Consider using Automated Item Generation (AIG) as an easy and cost-effective way to expand your item pool.

3. Monitor the Web

It is the responsibility of every testing organization to do their best locating and removing any and all test content exposed online—before it causes irreparable damage. Web monitoring is the practice of monitoring common places on the internet for stolen or otherwise exposed test content. While patrolling the web may not detect the theft in action, it certainly detects its outcomes and provides the necessary information to take action.

With web monitoring, you can find if and where any of your test content has been stolen and shared online and contributed to examinee pre-knowledge. Even more, you can identify the scope and extent of the breach as well. Once you’ve done this, you can work to get that content removed. In all your exams, watermarking will also enable you to easily track the source of the exposed content, and even locate the exact test taker who stole and distributed your test items.

4. Analyze Test Results

In order to effectively stop prior knowledge of exam content from impacting your test scores, it is necessary to track down and invalidate the scores of those who use pre-knowledge to cheat. You can do this by analyzing your test results on a regular basis. Also known as data forensics, this analysis enables you to find clues about whether (and where) cheating and theft are occurring. You can then cross-examine these data with your web monitoring efforts, along with other data from your test takers, to see who may have used illicit study material.

Yes, data analyses can help you determine whether examinees have seen test content ahead of time and are using pre-knowledge to gain an advantage. In addition, you can locate individuals who memorized your answers or were fed test content, and even locate those who shared content with others.

This four-pronged process enables you to stop examinees from using pre-knowledge. Some of these steps are designed to prevent pre-knowledge from happening in the first place, and others are there to catch individuals who have already used pre-knowledge. A final benefit can be utilized with this four-pronged approach as well—deterring cheating and theft by publicizing these test security measures.

In all, the combination of all four of these preventative, detective, and deterrent tactics is necessary to stop pre-knowledge from affecting your testing program. If your goal is to protect the validity of your exam scores, eradicating examinee pre-knowledge needs to be top of mind.

 

Alison Foster

View all articles

About Caveon

For more than 18 years, Caveon Test Security has driven the discussion and practice of exam security in the testing industry. Today, as the recognized leader in the field, we have expanded our offerings to encompass innovative solutions and technologies that provide comprehensive protection: Solutions designed to detect, deter, and even prevent test fraud.

Topics from this blog: Braindumps Test Security Consulting Data Forensics Exam Development K-12 Education Web Monitoring Test Security Basics Detection Measures Prevention Measures Certification Test Security Plan Higher Education Online Exams Automated Item Generation (AIG) Medical Licensure