Measuring conceptual understanding: the case of fractions

Developing measures of the quality of understanding of a given mathematical concept has traditionally been a difficult and resource-intensive process. We tested an alternative approach, called Comparative Judgement (CJ), that is based not on psychometric instruments or clinical interviews but collective expertise. Eight mathematics education experts used CJ to assess 25 student responses to a test designed to probe conceptual understanding of fractions. Analysis revealed the CJ assessment process yielded high internal consistency, inter-rater reliability and validity. We discuss the implications of the results for using CJ to measure mathematical understanding in a variety of domains and contexts.