LFI(2019)-manuscript+supp.pdf (1.16 MB)

Rigorous large-scale educational RCTs are often uninformative: Should we be concerned?

Download (1.16 MB)
journal contribution
posted on 05.02.2019, 16:42 by Hugues Lortie-Forgues, Matthew Inglis
There are a growing number of large-scale educational Randomized Controlled Trials (RCTs). Considering their expense, it is important to reflect on the effectiveness of this approach. We assessed the magnitude and precision of effects found in those large-scale RCTs commissioned by the EEF (UK) and the NCEE (US) which evaluated interventions aimed at improving academic achievement in K-12 (141 RCTs; 1,222,024 students). The mean effect size was 0.06 standard deviations (SDs). These sat within relatively large confidence intervals (mean width 0.30 SDs) which meant that the results were often uninformative (the median Bayes factor was 0.56). We argue that our field needs, as a priority, to understand why educational RCTs often find small and uninformative effects.

History

School

  • Science

Department

  • Mathematics Education Centre

Published in

Educational Researcher

Citation

LORTIE-FORGUES, H. and INGLIS, M., 2019. Rigorous large-scale educational RCTs are often uninformative: Should we be concerned?. Educational Researcher, 48 (3), pp.158-166.

Publisher

SAGE Publications © AERA

Version

AM (Accepted Manuscript)

Publisher statement

This work is made available according to the conditions of the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) licence. Full details of this licence are available at: https://creativecommons.org/licenses/by-nc-nd/4.0/

Acceptance date

26/01/2019

Publication date

2019

Notes

This paper was published in the journal Educational Researcher and the definitive published version is available at https://doi.org/10.3102/0013189X19832850.

ISSN

0013-189X

eISSN

1935-102X

Language

en

Exports

Loughborough Publications

Keywords

Exports