File(s) under embargo

Reason: Publisher requirement

34

days

23

hours

until file(s) become available

Measuring conceptual understanding in randomised controlled trials: can comparative judgement help?

journal contribution
posted on 09.05.2019 by Ian Jones, Marie-Josee Bisson, Camilla Gilmore, Matthew Inglis
An impediment to conducting high-quality quantitative research studies in education is the paucity of valid measures of learning gains. Studies often seek to investigate students’ deep, conceptual understanding yet many measures assess only surface, procedural understanding. One reason is that the development of validated measures of conceptual understanding is resource intensive, time consuming, and success is not guaranteed. We evaluated a novel and efficient technique, based on comparative judgement, for assessing conceptual understanding. We applied the technique to a randomised controlled trial in which students were taught simple algebra based on either the Grid Algebra or the MiGen software package. The participants were Year 5 students (N = 188) drawn from four primary schools who had not encountered algebra previously. An instrument from the literature (Concepts in Secondary Mathematics and Science: Algebra Scale), and a novel comparative judgement assessment were administered following the intervention. Students in the Grid Algebra condition outperformed those in the MiGen condition on both post-test measures. The comparative judgement technique performed similarly to the standard instrument but was far more efficient to design and implement. The technique can, in principle, be quickly applied to any target concept of interest. We conclude that comparative judgement is a valid, reliable and practical tool that could help to increase both the quantity and quality of quantitative research in education.

Funding

This work was funded by a Nuffield Foundation grant, a Royal Society Shuttleworth Education Research Fellowship awarded to IJ, a Royal Society Dorothy Hodgkin Fellowship awarded to CG, and a Royal Society Worshipful Company of Actuaries Education Research Fellowship awarded to MI.

History

School

  • Science

Department

  • Mathematics Education Centre

Published in

British Educational Research Journal

Volume

45

Issue

3

Pages

662-680

Citation

JONES, I. ... et al, 2019. Measuring conceptual understanding in randomised controlled trials: can comparative judgement help?. British Educational Research Journal, 45 (3), pp.662-680.

Publisher

Wiley © British Educational Research Association

Version

AM (Accepted Manuscript)

Publisher statement

This is the peer reviewed version of the following article: JONES, I. ... et al, 2019. Measuring conceptual understanding in randomised controlled trials: can comparative judgement help?. British Educational Research Journal, 45 (3), pp.662-680, which has been published in final form at https://doi.org/10.1002/berj.3519. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions.

Acceptance date

11/04/2019

Publication date

2019-06-01

ISSN

0141-1926

eISSN

1469-3518

Language

en

Exports

Logo branding

Exports