The effectiveness of a knowledge-based system as a simulation front-end

This research has shown that a knowledge-based system is an effective tool to help novice simulation users interpret and understand simulation output. The thesis describes the development and empirical evaluation of the prototype. A simulation program which adopts the discrete-event simulation approach simulates the behaviour of a local area network protocol, i.e., the Ethernet, with different sets of parameter values. The knowledge-based system carries out the 'analysis' of the simulation output covering the protocol efficiency and throughput. The knowledge-based system summarises the simulation output and upon request from the user, provides explanations to a conclusion arrived at. The summary is the relationship between any pair of variables; and the explanation is the justification as to how the pair are related. The strategy for building the knowledge base using production rules is also elaborated. There are different functions performed by the different sets of rules (or rule-sets). Their major functions, In parallel with the development objective, are interpreting numerical data, presenting output to users and providing explanations interactively. The rules are grouped accordingly to make the knowledge bases easier to maintain. In the explanation aspect, the few approaches attempted by other researchers to improve expert system explanation is discussed. It is argued that a mere regurgitation of 'fired' rules to explain the Ethernet behaviour is not adequate in this case. To circumvent this problem, a 'constructive' approach to explanation is employed. The explanation procedure rewrites the 'fired' rules in a more understandable form than the if-then rules. Unnecessary parts of the rules are ommitted to make the explanations clearer. Finally, an experiment carried out to evaluate the effectiveness of the prototype is described in detail. The effectiveness is measured from a few different perspectives. These are test scores, completion time for the test and the users' degree of confidence, both in the interpretation and explanation tasks. The results show that although some responses are mixed, there is evidence to suggest that the knowledge-based simulation system environment is beneficial to the target users.