CAA of Short Non-MCQ Answers

This paper presents a new approach for the computer-assisted assessment (CAA) of non- multiple choice questions (Non-MCQ) type and short answers given by students. The technique is developed for the assessment of text contents of free text answers to questions of factual disciplines. The Automated Text Marker (ATM) prototype automatically breaks down an expertly written model answer, to a closed-ended question, into the smallest viable unit of concepts with their dependencies accounted for by automatically tagging the resultant concepts and their dependencies with numbers. The same process is applied to each student’s answer and the resultant concepts and their dependencies are then pattern-matched with those of the model examiner’s answer. Two main components of ATM are the syntax and semantics analysers. In a prototype test, ATM provides for one score for the grammars and the other for the text contents. The focus of this paper is on semantic analysis of text contents since the syntactic analysis of sentences has been generally and successfully automated. Various examples of sentences of different factual disciplines such as those of Prolog programming, psychology and biology-related fields are analysed. Justifications for these analyses of sentences are provided and the corresponding prototype tests are conducted. The expected results from prototyping using ATM are obtained, indicating the reliability and feasibility of this new approach for the detailed assessment of text contents incorporating word order. Work is currently underway for building a larger and more comprehensive ATM system for analysing and assessing text components larger than sentences such as paragraphs and whole text passages. Unlike existing computerised assessment systems, ATM is not a predictive system, although, like a human assessor, it is not perfect.