Join Us at ATP for a Thought-Provoking & Action-Oriented Session on Improving Fairness in Testing

We've never been more excited for a presentation.

At ATP's Innovations in Testing 2022 conference, Caveon's David Foster will be presenting with four other industry-renowned experts on the topic of fairness in testing:

Item and Test Designs to Enhance Fairness and Inclusivity

Join Kristen Huff of Curriculum Associates; Maria Elena Oliveri of the Buros Center for Testing, University of Nebraska Lincoln; Stephen G. Sireci of the University of Massachusetts Amherst; and David Foster of Caveon for this groundbreaking Platinum Session.

About the Session

The design of tests and other assessments must be thoughtfully considered to meet particular purposes, and must do so in a way that is inclusive of the diversity inherent in society. Testing has converted to computerized administration over the past 30 years; however, tests today predominantly remain static, and they continue to use multiple-choice items. These are elements of test design that perpetuate early 20th-century ideals, reflecting a “white-dominant” culture of the day. This century-old concept of what tests should be results in differential test-taking experiences and performances across groups defined by poverty/privilege, ethnicity, language, and culture. The good news is, with the amazing 21st-century technology at our fingertips, tests can now be designed, built, and administered in ways that explicitly remove the sources that support this long-standing unfairness.

Two examples of such sources include (a) the ability of some individuals to cheat on tests, and (b) the ability of some individuals to utilize highly practiced test-taking skills. Both these strategies invalidly impact test scores; however, little has been done in item and test design to mitigate these effects. Clearly, cheating and coaching correlate with privilege, and so reducing the ability to cheat or use “testwiseness” skills to raise test scores will increase fairness (i.e., removing the advantage for privileged groups that use these tactics). Another example is the inflexibility of today’s testing conditions to allow for examinee choice in personalizing the testing experience.

This session will describe and demonstrate test (and item) design features that can be added or manipulated to specifically (a) reduce test score inflation due to both cheating and testwiseness, and (b) allow personalization of the test to promote culturally-sustaining practices. 

This session will include three presentations and a discussant, outlined below.

Sessions and Presenters

1. Using Technology to Give Tests and Items That Enhance Fairness and Inclusivity

David Foster, Caveon

The first presentation in this platinum session will illustrate how new test and item designs can be applied to any important testing program to reduce irrelevant inflation of test scores, such as that added by cheating and testwiseness

2. Empowering the Marginalized Through Algorithmic Item Design and UNDERSTANDARDized Test Administration

Stephen G. Sireci, University of Massachusetts Amherst

In the second presentation, Stephen Sireci from the University of Massachusetts will show how the concept of culturally-sustaining testing practices can be built into test design by allowing examinee choice, or a priori tailoring, of test context for each examinee; and by allowing examinees to draw from their funds of knowledge while taking an assessment.

3. Rolling Up Our Sleeves and Opening Our Hearts: Real Work in Inclusive Assessment Design

Kristen Huff, Curriculum Associates

The third presentation by Kristen Huff of Curriculum Associates will report on a concentrated effort to develop culturally inclusive, responsive, and sustaining test items on a large-scale K-12 diagnostic assessment.

4. A Discussion Among Presenters and Attendees

Maria Elena Oliveri, Buros Center for Testing, University of Nebraska Lincoln

A discussant, Maria Elena Oliveri, who works in the area of culturally-sustaining principled test design at the Buros Center for Testing, will provide commentary on the presentations and their relevance to improving fairness in testing.

Finally, we want to hear your thoughts. The session will conclude with a discussion among presenters and attendees as to how such design improvements might be practically used and how they specifically address the topics of systematic racism and general unfairness promoted by traditional test design.

Objectives & Learning Outcomes

The objectives and learning outcomes for this session are as follows:

1. Attendees will learn how traditional test and item design, supported by outdated notions of standardized testing, negatively impact test scores of individuals in particular groups in the testing population.
   
   
2. Attendees will learn how changes to test and item design can remove unwanted sources of systematic bias from test scores, such as cheating and testwiseness, resulting in an inclusive opportunity to prove abilities.
   
   
3. Attendees will learn how to design tests that incorporate examinee choice to personalize and make the testing experience more culturally relevant.
   
   
4. Attendees will understand the critical role of technology in this endeavor through descriptions and demonstrations.

We hope you join us for this Platinum Session at ATP, March 20-23, 2022!