A conceptual model framework for XAI requirement elicitation of application domain system

The use of data analytics and Machine Learning (ML) branches of AI for predictive and analytic knowledge retrieval has surged significantly in various industries (e.g., health, finance, business, and manufacturing). However, the acceptance of AI has been hindered by opaque models that lack transparency. Explainability in AI (XAI) has gained significant prominence owing to its focus on introducing avenues of accountability in AI. XAI acknowledges the importance of human factors and strives to incorporate them into the design process, recognising that the cognitive effort involved in understanding explanations is a key aspect. Mental Models play a crucial role in the XAI evaluative premise, but their current utility is limited. By intentionally designing explanations that align with users’ mental models, their experiences can be significantly enhanced, leading to improved understanding, satisfaction, trust, and performance. This study proposes using Mental Models to elicit explainability requirements and to develop an Ontology-Driven Conceptual Model to facilitate the learning process for a better understanding of explanations.