What factors affect student opinions of computer-assisted assessment?

During the introduction of computer-assisted assessment in a number of first- and second-year modules, we have been monitoring student performance and asking students to evaluate the use of on-line examinations. Initial results (Ricketts & Wilks, in press) suggested that both student performance and student opinions were strongly affected by the on-screen style of the assessment. We standardised the style of the assessment interface and continued the evaluation. Modules that involved the teaching of statistics to first-year students in three different subject areas (Biology, Business and Geography) all used on-line examinations, as did a second-year Computing module. We were surprised to find that students from one subject area found the use of on-line examinations considerably less acceptable than the other subjects. Only 55% of students in Geography preferred on-line examinations compared with 72% to 90% in the other subject areas. We have considered a number of possible explanations for the difference between the subject areas: a difference in preparedness for the on-line examination; a difference in performance; a difference in IT skills. As part of the evaluation we asked students about how well they were prepared for the on-line examinations, and whether they revised differently. Although there were differences in the responses in the four subject areas, this was not related to the acceptability of the on-line assessment. As part of the evaluation of the effectiveness of computer-assisted assessment we monitored student performance in successive years. In three of the subject areas there was an improvement and in the fourth there was a poorer performance, but the poorer performance did not occur in the group which found on-line assessment least acceptable. We compared the opinions of the students in the three first-year subject areas with second-year students of Computing, who have more highly developed IT skills. The three first-year subject areas all have similar development of IT skills, and if poor IT skills was an issue we would expect all three groups to have a lower opinion of on-line assessment than the second-year Computing students do. This was not the case, with the second-year Computing students showing no greater preference for on-line testing than the two subject areas which liked it most. This paper asks whether we should be concerned that student opinions of computer-assisted assessment may vary between cohorts and subject areas.