So, Your School Was Flagged by an Erasure Analysis, Now What? — Caveon

What Is an Erasure Analysis?

An erasure analysis (also known as an answer change analysis) is a data forensics analysis that examines the number and nature of erasure marks on answer documents (for paper-and-pencil tests) or logged answer changes (for computer-based tests). Improbable answer change patterns (e.g., a large number of wrong-to-right erasures/answer changes and/or clusters of tests with high rates of answer changes), particularly when associated with performance increases, can indicate possible coaching of students by teachers and/or tampering with test materials after the test. 

A lot of attention has been focused on erasure analyses and answer change analyses. Investigations triggered by these analyses have been publicized widely. People are asking, “Could this happen here, in my community?”

The pressure to cheat is real. In a self-selected Michigan survey, 30% of educators said they felt pressure to cheat and 8% actually cheated. We can’t generalize the result. But, it confirms what we already knew: A small percentage of educators have cheated and will continue to cheat. Hence, it is vital that departments of education pay attention to these and other statistics and heed warning signs of potential test security problems.

Can an Erasure Analysis Confirm Cheating?

An answer change analysis can provide important clues, but by itself, it cannot confirm cheating. Usually, erasure analyses will flag a school or classroom if its average number of wrong-to-right changes exceeds the state-wide average by three or four standard deviations. (You can learn more about the proper place for erasure analyses in this article.) The following observations seem relevant:

1. Changes on answer sheets are infrequent. Actual frequencies vary by state and grade, but most answer sheets will not have any erasures or logged changes, and very few will have more than four or five erasures or logged changes. This means that just a few answer sheets with a large number of changes are very unusual.
   
   
2. A lot of answer changes are NOT the same thing as cheating. Like with identical exam answers, the statistics may identify schools or classrooms that merit in-depth inspection. The statistics may indicate very low probabilities of “normal test taking.” But, we must NOT interpret erasure or answer change probabilities as the probability that tampering occurred. Doing so is fallacious and improper.
   
   
3. Investigations that are triggered by erasure and answer change analyses will continue. They will become commonplace and routine. We should not infer guilt from an investigation being conducted. Guilt or innocence is only inferred after the investigation has run its course.
   
   
4. A single statistic, such as the average number of wrong-to-right erasures or logged answer changes in a classroom, cannot tell the entire story. For example, educators at one school said the data were spurious because the average number of wrong-to-right answer changes was only 2 erasures higher than the state average. It might be true, if every answer sheet had 2 erasures. But, changing a lot of answers on only one-fourth of the answer sheets could have produced the same statistical flag.

What to Do if Your School Is Flagged by an Answer Change Analysis

If your school has been flagged by an erasure analysis, it is important to cooperate with the investigators. The flag was probably caused by something different about your school, but not tampering. The fact is that most investigations triggered by an answer change analysis do not uncover wrongdoing by educators.

Perhaps our unease with these analyses is due to the name we attach to the fact-finding that occurs after an answer change analysis occurs. If we were to refer to it as an “audit” instead of as an “investigation,” we might view it as routine and necessary instead of shocking and salacious. You can learn more about testing irregularities and how security investigations fit in with this article.