Using Rasch analysis to identify uncharacteristic responses to undergraduate assessments

Rasch Analysis is a statistical technique that is commonly used to analyse both test data and Likert survey data, to construct and evaluate question item banks, and to evaluate change in longitudinal studies. In this article, we introduce the dichotomous Rasch model, briefly discussing its assumptions. Then, using data collected in an example-generation task with undergraduate mathematics students, we use Rasch analysis to (1) rank the task questions and consider this ranking in relation to students’ conceptions and (2) identify those students whose responses misfit the model (e.g. students who score highly but get easier questions wrong). We conclude that because these responses misfit the Rasch model they are worthy of further (qualitative) examination, for instance to aid the discovery and classification of students’ misconceptions relative to formal theory.