Through appropriate exam design and statistical analysis, instructors can minimize variation in exam difficulty while maintaining an appropirate level of randomization to mitigate cheating.
Randomization has been shown to reduce collaborative cheating in assessments. However, randomization introduces the possibility that students may receive exams with varying levels of difficulty. In this study, researchers from the University of Illinois sought to identify the appropriate balance between exam fairness and exam security (randomization). The authors developed an algorithm to identify exams that were considered at the edges of the distribution of difficulty. The authors removed such exams and replaced them with newly randomized exams that fell within their predetermined appropriate range of difficulty. This approach reduced the variance of exam difficulty between exams. By so doing, the authors were able to filter out exams that were more difficult without compromising the level of randomization or exam security up until a certain point, based on a calculated measure of entropy. After that point, the ability to randomize questions and maintain fairness decreased substantially. This implies that within a certain range, exam fairness can be maintained without sacrificing the level of randomization.
Randomization introduces challenges with fairness but these can be overcome through appropriate statistical techniques.
Instructors need to understand the relationship between fairness and security when creating and analyzing their exams.