Extending flexibility in an existing on-line assessment system

This paper seeks to extend the work of seventeen years of research and development in the field of computer aided assessment. Work began on the Computer Aided Learning in Mathematics (CALM) Project in 1985 at Heriot-Watt University (http://www.calm.hw.ac.uk/). But, more general issues of automatic assessment are now being considered in collaboration with UK Examination Boards, commercial companies (http://www.i-assess.co.uk/) and Scottish academics through the forum of the Scottish Centre for Research into On-Line Learning and Assessment (SCROLLA) (http://www.scrolla.ac.uk/). To set the debate in context some of the main results of the CALM Project will be briefly reviewed. Two of the areas of research in SCROLLA are investigation into automatic assessment of higher order skills (Beevers et. al. 2003) and how traditional paper based questions translate into an equivalent on-line version using existing question types (Fiddes et. al. 2002). For example, the rewording of a question to allow it to be delivered in an assessment engine can mean that the student is provided with additional information that may not be available in the traditional paper based version, or the rewording may change the skills that are being tested. Some of these issues can be addressed by creating new question types to increase the flexibility given to the author in creating questions, and to increase familiarity of the students in the responses they can provide. The assessment system CUE has been used in UK programs such as SCHOLAR (http://scholar.hw.ac.uk/), which provides a wide range of questions in a variety of subjects (Higher, Advanced Higher and A Level in Biology, Chemistry, Computer Science, Mathematics and Physics). This provides a stable base from which CUE can expand. SCROLLA intends to provide CUE as a research and development resource for education institutions in Scotland from schools through to higher education over a range of subject areas. CUE provides an opportunity for exploring the research issues further and investigating possible solutions that more flexible question types may provide. Using examples chosen from Computer Science and Mathematics this paper will illustrate where some of the restrictions with currently available question types occur, and offer potential solutions for discussion and comment.