Peer assessment of mathematical understanding using comparative judgement

It is relatively straightforward to assess procedural knowledge and difficult to assess conceptual understanding in mathematics. One reason is that conceptual understanding is better assessed using open-ended test questions that invite an unpredictable variety of responses that are difficult to mark. Recently a technique, called comparative judgement, has been developed that enables the reliable and valid scoring of open-ended tests. We applied this technique to the peer assessment of calculus on a first-year mathematics module. We explored the reliability and criterion validity of the outcomes using psychometric methods and a survey of participants. We report evidence that the assessment activity was reliable and valid, and discuss the strengths and limitations, as well as the practical implications, of our findings.