Fission multimodale pour les systèmes d'interaction

Abstract

Computing systems are the outcome of scientific needs. Their popularity is due to their use by a great number of people. This fact led researchers to develop systems according to the user’s need and spread their use on a high scale. The current technological advancement has created the need to produce more powerful machines, easy to use and to meet the needs of users. To achieve these objectives, these machines should interfere in a harmonious way with the user. This could be made possible only if the systems are able to understand human communication. Human communications is conducted via many naturel modalities such as speech, gesture, eyes gaze and facial expressions. Inspired by human communications, multimodal systems were developed to combine many modalities according to the task, preferences and communication intentions. In this context, this thesis deals with the theme of multimodal fission for interaction systems. Our research is based mainly on 3 objectives. 1) We proposed a useful architecture for a multimodal system. This architecture is modeled, formally specified and refined by the use of colored Petri nets. It performs a multimodal fission module. 2) We modeled an ontological model i) that describes the environment of the multimodal system and ii) contains various applicable scenarios. These scenarios are stored as a pattern. Therefore, our fission algorithm is based on using the technique of pattern. We defined two patterns a) fission pattern to select the elementary subtasks for a complex command, and b) modality pattern: to associate each subtask to the appropriate modality (ies). Finally, 3) we proposed a new method / technique based on context using the Bayesian network to solve the problem of ambiguity or uncertainty in a system of multimodal fission. These techniques have been validated through case studies and using colored Petri nets and a simulation tool CPN-Tools. Thus, two real applications have been implemented 1) interface for the control of a robot that can be used to assist disabled or elderly users. This interface is implemented to validate the use of pattern technique in the fission process. 2) A GPS interface that indicates the itinerary for a car driver. This interface is implemented to validate our new method based on the context using the Bayesian network in the case of ambiguity or uncertainty.

Date	4 Feb 2014
Original language	French
Awarding Institution	École de technologie supérieure
Supervisor	Chakib Tadj (Supervisor) & Amar Ramdane-Cherif (Co-supervisor)