IJRUME_2015_preprint.pdf (984.87 kB)
Download file

Measuring conceptual understanding using comparative judgement

Download (984.87 kB)
journal contribution
posted on 03.03.2016, 11:48 by Marie-Josee Bisson, Camilla GilmoreCamilla Gilmore, Matthew InglisMatthew Inglis, Ian JonesIan Jones
The importance of improving students’ understanding of core concepts in mathematics is well established. However, assessing the impact of different teaching interventions designed to improve students’ conceptual understanding requires the validation of adequate measures. Here we propose a novel method of measuring conceptual understanding based on comparative judgement (CJ). Contrary to traditional instruments, the CJ approach allows test questions for any topic to be developed rapidly. In addition, CJ does not require a detailed rubric to represent conceptual understanding of a topic, as it is instead based on the collective knowledge of experts. In the current studies, we compared CJ to already established instruments to measure three topics in mathematics: understanding the use of p-­‐values in statistics, understanding derivatives in calculus, and understanding the use of letters in algebra. The results showed that CJ was valid as compared to established instruments, and achieved high reliability. We conclude that CJ is a quick and efficient alternative method of measuring conceptual understanding in mathematics and could therefore be particularly useful in intervention studies.


The studies reported in this manuscript were funded by a Nuffield Foundation grant.



  • Science


  • Mathematics Education Centre

Published in

International Journal of Research in Undergraduate Mathematics Education


BISSON, M-J., 2016. Measuring conceptual understanding using comparative judgement. International Journal of Research in Undergraduate Mathematics Education, 2 (2), pp.141-164.


© Springer


AM (Accepted Manuscript)

Publisher statement

This work is made available according to the conditions of the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) licence. Full details of this licence are available at: https://creativecommons.org/licenses/by-nc-nd/4.0/

Publication date



The final publication is available at Springer via http://dx.doi.org/10.1007/s40753-016-0024-3.