Performance-based item analysis for profiling skills and competencies for progress files

The University of Luton has successfully implemented Personal Development Planning (PDP) modules for students on HND 1 and level 1degree programmes and is in the process of introducing PDP at degree level 2. Computer-based assessment (CBA) at the University contributes to assessment practices and forms a part of the teaching, learning and assessment strategy. CBA is used to deliver approximately 7,000 tests per annum, across a range of disciplines, subjects and levels. This paper presents the findings of an exploratory item analysis into student performance on two PDP modules and a Scheme1 module assessed by CBA. The purpose is to investigate if CBA holds the potential to fulfil a diagnostic function over and above its role in summative assessments. Tracking progress, monitoring performance and measuring improvement over time forms a critical part of a personal development plan. The reporting functionality of most CBA software generates data that can inform on the performance of candidates and the educational validity of test items. The focus of this study is on the performance of stronger and weaker candidates on test items across differing question types. The underlying notion being that different question types might measure or signal a particular cognitive skill or competency level. With this in mind one might reasonably expect weaker candidates to perform at the lower cognitive levels and that maybe this is reflected in their performance on certain question types, as compared to the performance of stronger candidates on the same test items.