The World's Only Test Security Blog
Pull up a chair among Caveon's experts in psychometrics, psychology, data science, test security, law, education, and oh-so-many other fields and join in the conversation about all things test security.
Posted by Alison Foster
updated over a week ago
Automated Item Generation (AIG) is a process that leverages test item templates and computer algorithms to quickly create a large variety of item permutations, or test items. The result is hundreds or thousands of new test questions based on one single item model.
As the name suggests, AIG automates much of the effort involved in item creation—which is one the most time-intensive and costly aspects of test development. AIG helps to maintain and improve test quality in a world where online braindumps and content sharing platforms disclose test content and threaten test validity. With AIG, stolen test questions can be replaced quickly, and the damage that shared test content can inflict on assessment programs is palliated.
While many of the negative effects of cheating and content theft can be mitigated by a large enough item bank, it isn’t feasible for testing programs to dedicate teams of item writers, psychometricians, and subject matter experts to writing thousands of items. AIG technology is a great alternative for producing large numbers of high-quality test items. For more information, you can view a large body of scholarly research on the topic of AIG, including in this article by Gierl, Lai, and Turner (2012). While there are variations, the steps of automated item generation are generally as follows:
AIG has many short- and long-term benefits for testing organizations. From a bird’s eye view, the overall benefits of AIG include:
If security is your testing program’s priority (and it should be), then AIG has a lot to offer. An expanded item pool has many added benefits that impact overall test security endeavors:
AIG reduces development costs, increases the productivity of subject matter experts, and eliminates the expense required from redevelopment and republishing after content exposure. And those ripple into other benefits:
Here is an infographic that illustrates the lifetime cost of AIG compared to the cost of traditional item development (click to enlarge):
AIG has been used by large testing programs for many years. However, the plausibility of AIG has been out of reach for smaller testing programs with limited budgets and slimmer teams. Caveon AIG aims to change that. Available through Scorpion™, Caveon’s state-of-the-art AIG tools are built right into the software. Caveon AIG also utilizes SmartItem™ technology for even greater test security and item protection. Since AIG tools aren't all created equal, it’s important you put time and effort into researching the best AIG tool for your specific program. Look for AIG tools that include the following features:
Automated item generation leverages test item templates and computer algorithms to quickly create hundreds or thousands of new item permutations based on one single item model.
AIG has many short- and long-term benefits for testing organizations. These benefits include heightened security by increasing item pool size and reducing item exposure; cost savings through reduced redevelopment costs and increased item pool lifespan; time savings for subject matter experts and item creators; access to certain test designs including CATs and LOFTs that require large item banks; and more.
AIG is becoming more accessible for smaller testing programs and programs with limited resources, but not all AIG tools are created equal. It is important to find an AIG tool that is easy to use and offers psychometric transparency, increased item design capability, and ample software integration options. Choose the software that best aligns with the needs of your test program
For more than 18 years, Caveon Test Security has driven the discussion and practice of exam security in the testing industry. Today, as the recognized leader in the field, we have expanded our offerings to encompass innovative solutions and technologies that provide comprehensive protection: Solutions designed to detect, deter, and even prevent test fraud.