Baruah_N_Gill_M_Greenhow_M_b1.pdf (155.5 kB)
Issues with setting online objective mathematics questions and testing their efficacy
conference contribution
posted on 2009-04-23, 12:50 authored by N. Baruah, Mundeep Gill, Martin GreenhowThe Mathletics database now comprises many mathematical topics from
GCSE to level 2 undergraduate. The aim of this short paper is to document,
explore and provide some solutions to the pedagogic issues we are facing
whilst setting online objective questions across this range. Technical issues
are described in the companion paper by Ellis, Greenhow and Hatt (2006).
That paper refers to “question styles to stress that we author according to the
pedagogic and algebraic structure of the content of a question; random
parameters are chosen at runtime ... This results in each style having
thousands, or even millions, of realisations seen by the users.” With this
emphasis, and with new topics being included, new question types beyond
the usual multi-choice (MC) etc have been developed to ask appropriate and
challenging questions. We feel that their pedagogic structure (and underlying
code) is widely applicable to testing beyond the scope of Mathematics. This
paper describes three of the new question types: Word Input, Responsive
Numerical Input and 4/True/False/Undecidable/Statement/Property. Of
generic importance is the fact that each of these question types can include
post-processing of submitted answers; sample Javascript coding that checks
the validity of the input(s) before marking takes place is described. In common
with most of the rest of the question style’s content this could be exported to
other CAA systems.
Ellis et al (2005) and Gill & Greenhow (2006) describe initial results of a trial
of level 1 undergraduate mechanics questions. This academic year we have
expanded the range of tests to foundation and level 1 undergraduate algebra
and calculus, involving several hundred students. First and foremost we have
underlined the value of Random Numerical Input (RNI) question types
compared with traditional Numerical Input (NI) types for which answer files
resulting from questions with randomised parameters are exceptionally
difficult to interpret. Despite our current lack of a consistent and fullymeaningful
way of encoding the mal-rules within the question outcome
metadata, mal-rule-based question types (MC, RNI etc) have been analysed
in terms of difficulty, discrimination and item analysis. In the case of multiplechoice
questions any weaknesses are separately identified as skill-based or
conceptual.
History
School
- University Academic and Administrative Support
Department
- Professional Development
Research Unit
- CAA Conference
Citation
BARUAH, N., GILL, M. and GREENHOW, M., 2006. Issues with setting online objective mathematics questions and testing their efficacy. IN: Danson, M. (ed.). 10th CAA International Computer Assisted Assessment Conference : Proceedings of the Conference on 4th and 5th July 2006 at Loughborough University. Loughborough : Lougborough University, pp. 55-70Publisher
© Loughborough UniversityVersion
- AM (Accepted Manuscript)
Publication date
2006Notes
This is a conference paper.ISBN
095395725XLanguage
- en