Measurement of factors determining relevance judgements

This study has focused on cognitive aspects of the human processes involved in relevance judgements. Several criteria and a great number of measures have been proposed and used for relevance assessment. However, there is a lack of agreement as to which are the best measures and to what extent they are affected by variability of relevance judgements. The purpose of this research was to identify those cognitive factors, which primarily contribute to relevance judgement. In tills study, sixteen criteria that influence cognitive relevance were identified and used. The study addressed three questions: (1) What cognitive factors affect relevance judgement? (2) What is the importance of each factor on relevance judgement? (3) Do the factors remain stable over time? Both qualitative and quantitative techniques were used for data collection and analysis. Structural Equation Modelling (SEM) as a multivariate technique was used to develop a statistical model of cognitive relevance. It seems, it is the first time that this technique has been applied to measure cognitive factors of relevance. Saracevic's (1996) stratified model, as a cognitive IR model was adopted to provide a necessary framework to incorporate relevance cognitive theory and a user approach in measuring relevancy. [Continues.]