Joined up? Or just lucky? Implementing CAA in Scotland

Scotland, with 1/10th of the UK population, has its own and distinct education system. It is funded through the Scottish Executive, based in Edinburgh, where the new Scottish parliament now sits. Each education sector is largely autonomous with separate funding and support agencies. The primary qualifications authority for the secondary and further education sectors is the Scottish Qualifications Authority (SQA). They hold a near monopolistic dominance of the qualifications framework in use in Scotland’s schools and colleges. There is a significant overlap in the provision of the schools and Further Education (FE) sectors who share a common unitised framework which spans both vocational and academic qualifications. Therefore, developments in one sector, has implications for the other. There are 46 Scottish FE colleges which have operated, since the early 1990s as self governing autonomous institutions centrally funded by the Scottish Further Education Funding Council (SFEFC). Approximately 1/3rd of HE in Scotland is delivered by the FE sector 60% of first entrants to HE do so at a FE college. In UK terms we have the highest HE participation rates. In the early years of the incorporated status of colleges an element of competition was apparent, although more recently there has been a return to a spirit collaboration to enhance the effectiveness of the sector. In late 2000, SFEFC awarded funding to Glenrothes College, along with a number of partner colleges, to investigate and develop a knowledge base in Computer Assisted Assessment (CAA). The authors were co-managers of the project titled ‘Best Practice in OnLine Assessment’ (BPOL), and have since become involved in a number of initiatives in Scotland developing Computer Assisted Assessment approaches. This paper outlines the activity of the project and its influence on subsequent developments in Scotland. Objective testing is rarely used for summative assessment within the Scottish FE system, or indeed, within the SQA qualifications framework at all. It is most often used in open or flexible learning materials as formative, self-assessment tasks. Historically, it has been effectively disallowed by the emphasis on assessment of vocational competence based upon criterion referenced, range based standards. Therefore, it was no surprise that there was very little existing practice in the use of CAA. Also, the capacity of college network infrastructures to support CAA was only then being realised. However, the BPOL project coincided with a shift by the SQA towards a more holistic approach to assessment which allowed sampling of learning outcome criteria and therefore objective testing with cut-off scores became a valid option for consideration. However, much work needs to be done by both the SQA and the FE sector to ensure that objective testing and its delivery by CAA approaches is a valid and robust methodology. It became clear that there were fundamental issues with regard to the way individual unit specifications (notional discrete ‘modules’ of learning within a subject), had been written. There was no shared understanding of a taxonomy of educational objectives to provide a theoretical underpinning of the description of a learning outcome. This did not present as a problem under the old assessment regime as an effective moderation system is in place which ensures national standards are achieved by all colleges. It is, however, a significant problem for those colleges pioneering CAA and the development of objective approaches. As interest in CAA gathered pace during the year 2002, a number of other projects were funded within Scotland. A multi-agency approach has evolved and includes the SFEFC, the SQA, the Scottish Executive, HEIs, the colleges, schools and other stakeholders. It has not been a planned and co-ordinated national approach but may yet result with joined-up processes of CAA development, and who knows – a model for future progress? This paper should be viewed as a ‘snap-shot’ of this progress achieved by Mid 2003.