Information retrieval from civil engineering repositories: the importance of context and granularity

Information about the design and construction of buildings can be structured in a particular way. This is especially correct given the increasing complexity of building product models and the emergence of building information models with project documents linked to them. In addition, engineers usually have distinct information needs. Research shows that engineers working with building information models place particular importance on the understanding of retrieved content before using it or applying it and that exploration of context is essential for this understanding. Both these factors (the nature of engineering content and the information needs of engineers) make general information retrieval techniques for computing relevance and visualizing search results less applicable in civil engineering information retrieval systems. This paper argues that granularity is a fundamental concept that needs to be considered when measuring relevance and visualizing search results in information retrieval systems for repositories of building design and construction content. It is hypothesized that the design of systems with careful regard for granularity would improve engineers’ relevance judgment behavior. To test this hypothesis, a prototype system, called CoMem-XML, was developed and evaluated in terms of the time needed for users to find relevant information, the accuracy of their relevance judgment, and their subjective satisfaction with the prototype. A user study was conducted in which test subjects were asked to complete tasks by using various forms of the prototype, to complete a satisfaction questionnaire, and to be interviewed. The findings show that users perform better and are more satisfied when the search result interface of the CoMem-XML system presents only relevant information in context. On the other hand, interfaces that present the retrieved information out of context (i.e., without highlighting its position in the parts hierarchy) are less effective for participants to judge relevance.