Exploring deictic interface referencing outside-vehicle landmarks: Study of in-cabin multimodal interaction

<p dir="ltr">Multimodal interfaces have the potential to enhance user experience by aligning with natural human communication. While previous In-Vehicle Infotainment (IVI) systems have explored voice and gesture for internal controls (e.g., using voice and gesture to operate car side windows), little research has examined how drivers might reference objects outside the vehicle. This study investigates the user experience of in-cabin deictic interfaces that enable multimodal referencing of external landmarks. Through a participatory design workshop and a user study, we explored how drivers use combined speech, gestures and visual cues to reference external landmarks during driving. Our preliminary findings provide early empirical insights into external-referencing HMI design, user behaviour and technology acceptance, informing design implications and challenges for future automotive systems that support outside-referencing interaction.</p>