Lead


Lay summary
MULTI : MULTImodal Interaction and MULTImedia Data Mining

As a unified research theme, the goal of the MULTI project is to carry out fundamental research in the field of multimodal interaction, mainly covering a wide range of critical activities and applications, including recognition and interpretation of spoken, written and gestural language, recognition and interpretation of visual scenes, and processing and indexing of multi-channel (e.g., audio-visual)data streams. Other key sub-themes of MULTI include the control of information access, typically through biometric user authentication techniques (including speaker and face verification). Mainly exploiting related technologies of statistical modeling and data mining, MULTI also investigates advanced approaches towards the structuring, retrieval and presentation of multimedia information (e.g., resulting of multimodal meeting or lecture recordings), which also represents a wide-ranging and important research area that includes not only the multimodal interaction aspects, but also multimedia document analysis, indexing, and information retrieval, thus involving complex computer vision and data fusion algorithms. The research areas addressed in MULTI thus cover 6 complementary research themes, namely: (1) machine learning, (2) speech and audio processing, (3) computer vision,(4) information retrieval, (5) biometric authentication, and (6) multimodal interaction.