Computer-based vs. paper-based testing: are they the same?

Comparability studies of paper-based tests and computer-based tests focused mainly on the equivalence of both forms and the contributing factors affecting this concept. There have been several studies in different fields such as education, psychology, ergonomics and L1 reading research (Sawaki, 2001). However, there has been no empirical study so far that has investigated the effect of prior computer familiarity on students’ performance taking L2 tests (Chapelle & Douglas, 2006). Chapelle & Douglas (ibid) also mentioned the significance and lack of differential validity studies and how motivating it is to find out more about performance on computer-based testing. Sawaki (2001) argues that this type of empirical work should employ different methodologies such as eye movement, verbal protocols, post hoc interviews, and questionnaires to reveal useful results. Thus, this ongoing study examines the comparability of paper based tests and the computer based testing in L2 reading context, and the impact of test takers' characteristics, i.e., computer familiarity, computer preference, gender and test taking strategies on students' performance on computer based tests, and sequentially on comparability with paper based tests. 167 Saudi medical students took three reading achievement tests on both paper and computer. The Test of English as a Foreign Language (TOEFL) was devised to measure the students' proficiency and anchor the study tests. The study questionnaires focused on demographic information, participants' computer familiarity and preference of testing mode. The interview examined any change of preference after exposure to CBT. A triangulation of think aloud reports and post hoc interviews were employed to gain insight into strategies used on both testing modes, and to confirm comparability of both modes for validity purposes. The results are likely to reveal some information about the equivalence of both testing modes based on a scientific systematic perspective and have implications for the implementation of computer based reading tests in the context of the medical faculty EAP programme.