LFI(2019)-manuscript+supp.pdf (1.16 MB)
Download file

Rigorous large-scale educational RCTs are often uninformative: Should we be concerned?

Download (1.16 MB)
journal contribution
posted on 05.02.2019, 16:42 authored by Hugues Lortie-Forgues, Matthew InglisMatthew Inglis
There are a growing number of large-scale educational Randomized Controlled Trials (RCTs). Considering their expense, it is important to reflect on the effectiveness of this approach. We assessed the magnitude and precision of effects found in those large-scale RCTs commissioned by the EEF (UK) and the NCEE (US) which evaluated interventions aimed at improving academic achievement in K-12 (141 RCTs; 1,222,024 students). The mean effect size was 0.06 standard deviations (SDs). These sat within relatively large confidence intervals (mean width 0.30 SDs) which meant that the results were often uninformative (the median Bayes factor was 0.56). We argue that our field needs, as a priority, to understand why educational RCTs often find small and uninformative effects.

History

School

  • Science

Department

  • Mathematics Education Centre

Published in

Educational Researcher

Volume

48

Issue

3

Pages

158 - 166

Citation

LORTIE-FORGUES, H. and INGLIS, M., 2019. Rigorous large-scale educational RCTs are often uninformative: Should we be concerned?. Educational Researcher, 48 (3), pp.158-166.

Publisher

SAGE Publications

Version

AM (Accepted Manuscript)

Rights holder

© AERA

Publisher statement

This paper was published in the journal Educational Researcher and the definitive published version is available at https://doi.org/10.3102/0013189X19832850.

Acceptance date

22/01/2019

Publication date

2019-03-11

Copyright date

2019

ISSN

0013-189X

eISSN

1935-102X

Language

en