Model-Based, Multimodal Interaction in Document Browsing.

Eslambolchilar, Parisa and Murray-Smith, Roderick (2006) Model-Based, Multimodal Interaction in Document Browsing. In: Machine Learning for Multimodal Interaction. Third InternationalWorkshop, MLMI 2006 Bethesda, MD, USA, May 1-4, 2006 Revised Selected Papers. Lecture Notes in Computer Science (LNCS), 4299/2006 . Springer Berlin / Heidelberg, pp. 1-12. ISBN 978-3-540-69267-6

[img] Download (400kB)
Official URL:

Share your research

Twitter Facebook LinkedIn GooglePlus Email more...

Add this article to your Mendeley library


In this paper we introduce a dynamic system approach to the design of multimodal interactive systems. We use an example where we support human behavior in browsing a document, by adapting the dynamics of navigation and the visual feedback (using a focus-in-context (F+C) method) to support the current inferred task. We also demonstrate non-speech audio feedback, based on a language model. We argue that to design interaction we need models of key aspects of the process, here for example, we need models for the dynamic system, language model and sonification. We show how the user's intention is coupled to the visualization technique via the dynamic model, and how the focus-in-context method couples details in context to audio samples via the language identifcation system. We present probabilistic audio feedback as an example of a multimodal approach to sensing different languages in a multilingual text. This general approach is well suited to mobile and wearable applications, and shared displays.

Item Type: Book Section
Additional Information: The original publication is available at
Keywords: Multimodal Interaction; Document Browsing; Continuous interaction; User behavioral models; Language identification system; Model-based interactive method; LNCS; MLMI’06; Hamilton Institute.
Academic Unit: Faculty of Science and Engineering > Computer Science
Faculty of Science and Engineering > Research Institutes > Hamilton Institute
Item ID: 1774
Identification Number: 10.1007/11965152_1
Depositing User: Hamilton Editor
Date Deposited: 11 Jan 2010 14:49
Publisher: Springer Berlin / Heidelberg
Refereed: No

Repository Staff Only(login required)

View Item Item control page

Document Downloads

More statistics for this item...