Implications of patterns of use of freely-available online formative tests for online summative tasks

The use of online assessment tasks in a summative context can create tensions between the institution’s need for security to ensure the validity of individual evaluations and the student’s need for flexibility of access. This is especially the case in recent years, with the upsurge of students engaged in paid employment while enrolled in full-time study. The lowest rate of engagement of students in paid employment at the three institutions in which our study was based was 65%, the highest 75%. One quarter of all students at this institution spent more than 20 hours a week in paid employment. Ninety seven percent of students in paid work were enrolled on a full-time basis. This study determined from automatically recorded times of logon, individual question submission and whole test submission the patterns of use of online feedback-enriched MCQ tests by 656 students across the three institutions in Perth, Western Australia. The conditions under which the tests were available to students varied from a strictly secured, summative task available for a limited time on campus within hours governed by the accessibility of automatically locked-down computer rooms and the availability of staff for live or video invigilation to a freely accessible formative learning exercise. Mismatches between preferred and available times severe enough to exclude some external students from assessment were identified. Evidence was found that for younger (16-18 year old) students especially, meaningful engagement with test-structured tasks lasts no more than 10 minutes, one third of the designed time of our current summative online tests. The one third, approximately, of enrolled students who did not use the online test facility had significantly poorer academic outcomes. The advantage granted by test use increased substantially with repetition. The question of how to ensure the security and validity of online testing while increasing real flexibility of access remains unresolved for us. We accept the social responsibility of finding a solution.