The World's Only Test Security Blog

Pull up a chair among Caveon's experts in psychometrics, psychology, data science, test security, law, education, and oh-so-many other fields and join in the conversation about all things test security.

The Incident of the Test Killer: A Case Study

Posted by Caveon

updated over a week ago

Data Forensics is essential for assessing the test security threats to your program and protecting the validity of your exams. In addition to being one of the best sources for discovering test security threats and weaknesses, data forensics provides the evidence you need for taking action when your exam's security has been breached.

The Summary

An incident has been uncovered in a classroom and you want to understand the truth about what occurred, and whether or not the incident was isolated or widespread.

Using Data Forensics, it is possible to illuminate which test results should be trusted, and which results may require additional scrutiny. 

The Situation

During the routine monitoring of a state-wide test, Caveon's data forensics team detected a school that had a cluster of tests with similar response patterns and a high pass rate.

The unusual results prompted the state education agency (SEA) to send in investigators, who discovered a “test killer” booklet of harvested questions and answers that had been used to artificially inflate some students’ scores. They confiscated the booklet, and pass rates returned to normal.

This was a test security success story, but Caveon's data scientists couldn't help but wonder: what would have happened if the test killer had never been found? Could they pinpoint wrongdoing on its own?

The Question

Caveon experts took up the challenge to find out if our data forensics could uncover the individual test killer responsible for the security breach using only data. The results were compelling.

By applying an iterative statistical model designed to analyze the differences and similarities between tests, Caveon experts not only extracted the answer key that had been shared, but also confirmed the individuals who had used it. Out of more than 23,500 tests, the data analysis identified nine tests that had a high probability of using the answer key.

The Impact

The Incident of the Test Killer shows that under appropriate conditions, data forensics can be used to find the source of exposed test questions, as well as the culprits who use it to inflate their scores.
The impact of this type of analysis is powerful. When Caveon analyzes clusters of extremely similar tests from a single location, a model of inappropriate test-taking may be developed. This model can then be used to help identify other candidates who gained an unfair advantage through pre-knowledge. 

Numbers don't lie. They reveal the truth.

About Caveon

For more than 18 years, Caveon Test Security has driven the discussion and practice of exam security in the testing industry. Today, as the recognized leader in the field, we have expanded our offerings to encompass innovative solutions and technologies that provide comprehensive protection: Solutions designed to detect, deter, and even prevent test fraud.

Topics from this blog: Data Forensics Investigating Security Incidents K-12 Education Detection Measures