final_inspire_april-14.pdf (96.86 kB)
The need for evidence innovation in educational technology evaluation
conference contributionposted on 2014-09-03, 12:54 authored by Melanie KingMelanie King, Ray DawsonRay Dawson, Firat BatmazFirat Batmaz, Steve Rothberg
More complex and chaotic methods are being adopted in the development of technology to enhance learning and teaching in higher education today in order to achieve innovation in teaching practice. However, because this type of development does not conform to a linear process-driven order, it is notoriously difficult to evaluate its success as a holistic educational initiative. It is proposed that there are five factors that impact on effective educational technology evaluation, which contributes to insubstantial evidence of positive outcomes, these being: premature timing; inappropriate software evaluation techniques and models; lack of shared understanding of the terminology or the semantics of education technology; the growing complexity of agile and open development; and the corporatisation of higher education. This paper suggests that it is no longer helpful for policy makers to evaluate whether educational technology project outcomes were successful or unsuccessful but instead they should use agile evaluation strategies to understand the impact of the product, process and outcomes in a changing context. It is no longer useful to ask the question, ‘did the software work?’ The key is for software developers and policy-makers to ask ‘what type of software works, in which conditions and for whom?’ To understand this, the software development community needs to look at adopting evaluation strategies from the social science community. For example, realist evaluation supplies context driven and evidence-based techniques, exploring outcomes that tend towards the social rather than technical. It centres on exploring the ‘mechanisms’, ‘contexts’ and ‘outcomes’ associated with an intervention and is a form of theory-driven evaluation that is the theory and reasoning of its stakeholders that is rooted in practitioner wisdom.
- University Academic and Administrative Support
- Centre for Engineering and Design Education (CEDE)
Published inInternational Conference on Software Process Improvement Research and Education (INSPIRE 2014) Proceedings of INSPIRE XIX: Global Issues in IT Education
Pages9 - 23 (15)
CitationKING, M ... et al., 2014. The need for evidence innovation in educational technology evaluation. IN: Uhomoibi, J. ... et al. (eds.) Proceedings of INSPIRE XIX: Global Issues in IT Education, Southampton, UK, 15 April 2014, pp. 9 - 23.
PublisherBritish Computer Society
- AM (Accepted Manuscript)
Publisher statementThis work is made available according to the conditions of the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) licence. Full details of this licence are available at: https://creativecommons.org/licenses/by-nc-nd/4.0/
NotesThis is a conference paper. It was presented at the 19th Annual INSPIRE conference.