Darby_halo influence.pdf (76.7 kB)
Download fileAre course evaluations subject to a 'halo effect'?
journal contribution
posted on 2007-07-20, 15:00 authored by Jenny A. DarbyMany course evaluations including those used in schools by OFSTED, and colleges and universities employ a
number of scales as a means of evaluating various aspects of the educational experience of the student. It tends to
be assumed students consider the scales independently. This paper argues students are influenced by a ‘halo
effect’ when making judgements about the merits of aspects of a course. In this study student evaluations were
obtained from 161 university lecturers attending probationary training courses. With a Likert style structured scale
a favourable evaluation on one aspect of a course was shown to be linked with favourable evaluations on other
aspects. Similarly with unfavourable evaluations different aspects were shown to correlate. This was not the case
with an open ended evaluation format. There were no links between reactions on the two types of evaluations.
Implications for the interpretation of course evaluations are discussed.
Student evaluations of courses using scaled responses are commonly used in school by OFSTED inspectors and
in colleges and universities as part of their external grading process. Research studies of course evaluations have
a long history. An early example by Bassin (1974) included a series of Likert scales concerned with five aspects
of teaching. These were lecture quality, exam quality, text suitability, participation and consideration. It was
found that instructors of quantitative courses received lower ratings than those of non-quantitative courses.
Pohlmann (1975) evaluated five aspects of courses, namely an overall view of how good the course was, how
interested the tutor was in the student, how difficult the student found the course, whether assignments were
clearly marked and how good the tutors actual presentation was. Pohlmann found undergraduate student’s
evaluations were better on elective than required courses. This use of scales has continued with Rae (1997 p
113-125), and Shevlin et al (2000) recommending using structured scales. These researchers in common with
many others assume the scales used are independent. This assumption is central to many course evaluations.
Different aspects of courses are usually compared by looking at mean scores of scales which are thought to be
independent. Sadly these may have little real value if the individual scores which make up those means are a
result of responses on one scale being influenced by those on another.
In an early review of the literature by Cohen (1981) there was evidence of some attempt to look at correlations
between evaluation scales. This tended to be limited to very specific and predicted areas such as that between a
favourable student rating of instructors’ skills and the student having received better grades. The issue of the
independence of scales in general has, been neglected in more recent texts on course evaluation methodology
(eg Holcomb 1998, Rae 2002, and Salas et al 2003), also in the more broadly based research methodology texts
(eg. Fowler 2002, Hayes 2000 and Shaughnessey et al 2000).
The present study questions the independence of measures on evaluation scales in the light of the ‘halo effect’,
which is well known in the field of person perception but is not a concept which has been commonly applied to
course evaluations. Blum and Naylor (1968 p. 200) see the ‘halo effect’ simply as the; ‘tendency to let our
assessment of an individual on one trait influence our evaluation of that person on other specific traits’. This
definition allows for any influence to be positive or negative.
History
School
- Social Sciences
Department
- Communication, Media, Social and Policy Studies
Citation
DARBY, J.A., 2007. Are course evaluations subject to a 'halo effect'? Research in Education, No. 77, pp. 46-55Publisher
© Manchester University PressPublication date
2007Notes
This article was published in the journal, Research in Education [© Manchester University Press].ISSN
0034-5237Language
- en