Caveon Security Insights Blog

The Incident of the Test Killer — Caveon

Written by Caveon | April 23, 2021 at 9:23 AM

Data Forensics is essential for assessing the test security threats to your program and protecting the validity of your exams. In addition to being one of the best sources for discovering test security threats and weaknesses, data forensics provides the evidence you need for taking action when your exam's security has been breached.

The Summary

An incident has been uncovered in a classroom and you want to understand the truth about what occurred, and whether or not the incident was isolated or widespread.

Using Data Forensics, it is possible to illuminate which test results should be trusted, and which results may require additional scrutiny. 

The Situation

During the routine monitoring of a state-wide test, Caveon's data forensics team detected a school that had a cluster of tests with similar response patterns and a high pass rate.

The unusual results prompted the state education agency (SEA) to send in investigators, who discovered a “test killer” booklet of harvested questions and answers that had been used to artificially inflate some students’ scores. They confiscated the booklet, and pass rates returned to normal.

This was a test security success story, but Caveon's data scientists couldn't help but wonder: what would have happened if the test killer had never been found? Could they pinpoint wrongdoing on its own?

The Question

Caveon experts took up the challenge to find out if our data forensics could uncover the individual test killer responsible for the security breach using only data. The results were compelling.

By applying an iterative statistical model designed to analyze the differences and similarities between tests, Caveon experts not only extracted the answer key that had been shared, but also confirmed the individuals who had used it. Out of more than 23,500 tests, the data analysis identified nine tests that had a high probability of using the answer key.

The Impact

The Incident of the Test Killer shows that under appropriate conditions, data forensics can be used to find the source of exposed test questions, as well as the culprits who use it to inflate their scores.
 
The impact of this type of analysis is powerful. When Caveon analyzes clusters of extremely similar tests from a single location, a model of inappropriate test-taking may be developed. This model can then be used to help identify other candidates who gained an unfair advantage through pre-knowledge. 

Numbers don't lie. They reveal the truth.