Project

Back to overview

Platform for Reproducible Acquisition, Processing, and Sharing of Dynamic, Multi-Modal Data

Applicant Liebling Michael
Number 164022
Funding scheme R'EQUIP
Research institution IDIAP Institut de Recherche
Institution of higher education Idiap Research Institute - IDIAP
Main discipline Information Technology
Start/End 01.07.2016 - 31.12.2017
Approved amount 205'217.00
Show all

All Disciplines (3)

Discipline
Information Technology
Electrical Engineering
Embryology, Developmental Biology

Keywords (10)

reproducible research; microscopy; data sharing; robotics; biometry; signal processing; image processing; machine learning; vision; imaging

Lay Summary (French)

Lead
La disponibilité de bibliothèques de sons, d'images et de films digitaux, charactérisés avec précision, sont une resource ensentielle pour faire avancer la recherche en traitement du signal, y compris pour le traitement des images biomédicales, la biométrie, la vision par ordinateur et l'intéraction homme-machine. Cependant, des bibliothèques ouvertes et bien charactérisées comportant des média liés à des sujets et des capteurs (micros, caméras) mobiles, en 2 et 3 dimensions (2D,3D) et couvrant une large gamme d'échelle (micro et macro) sont moins fréquentes. Ce projet cherche à combler cette lacune.
Lay summary
Contenu et objectifs du travail de recherche
Ce projet vise à doter l'Institut de Recherche Idiap d'une plateforme pour la collection, le traitement et la distribution de données son et image dynamiques qui couvriront des domaines d'application et échelles diverses. La particularité du système sera de permettre l'acquisition, de manière reproductible, de données à partir d'objets (sujets ou sources) et/ou de capteurs (microphones, caméras) en mouvement et de couvrir de manière systématique un large éventail de types de mouvements, d'angles de vue (en 2D ou 3D) et d'échelles. 

Concrètment, la plateforme comprendra une unité de stockage et de distribution, un noyau de calcul haute-performance pour traiter et distribuer les données, deux bras robotiques pour manipuler (de manière contrôlée et reproductible) des échantillons et capteurs en mouvement, ainsi qu'un microscope en fluorescence 3D de type light-sheet. 

Contexte scientifique et social du projet de recherche
Au delà de la microscopie (une nouvelle direction de recherche à l'Idiap) la plateforme sera suffisemment générique pour supporter à la fois la collection de types de données déjà bien établis à l'Idiap (parole, vision, biométrie), mais dans un contexte dynamique, et de stimuler des domaines nouveaux de recherche de traitement du signal en permettant aux sources et capteurs d'être en mouvement, permettant ainsi la capture de représentations réalistes monde réel tout en étant précisément reproductibles.
 
Direct link to Lay Summary Last update: 19.04.2016

Responsible applicant and co-applicants

Collaboration

Group / person Country
Types of collaboration
Willy Supatto / Ecole Polytechnique, Paris France (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Nadia Mercader / Uni Bern Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Henning Müller / HES-SO Sierre Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
- Research Infrastructure
Daniel Schorderet / IRO, Sion Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
- Research Infrastructure
B.S. Manjunath / University of California, Santa Barbara United States of America (North America)
- in-depth/constructive exchanges on approaches, methods or results
- Publication

Communication with the public

Communication Title Media Place Year
Media relations: radio, television L’Idiap à Martigny développe un nouveau système de prise d’images destiné à la recherche Canal 9 Western Switzerland 2017
Media relations: print media, online media Un robot créé par deux institutions valaisannes Le Nouvelliste Western Switzerland 2017

Associated projects

Number Title Start Funding scheme
172627 Robot skills acquisition through active learning and social interaction strategies (ROSALIS) 01.04.2018 Project funding (Div. I-III)
159227 Computational Methods for Temporal Super-resolution Microscopy 01.04.2016 Project funding (Div. I-III)
160811 SMILE: Scalable Multimodal sign language Technology for sIgn language Learning and assessmEnt 01.03.2016 Sinergia
140912 Object Detection with Active Sample Harvesting (DASH ) 01.10.2012 Project funding (Div. I-III)
180417 LEARNing physical manipulation skills with simulators using REAListic variations (LEARN-REAL) 01.04.2019 CHIST-ERA
179217 Computational biomicroscopy: advanced image processing methods to quantify live biological systems 01.04.2018 Project funding (Div. I-III)

Abstract

Large, diverse, and precisely-characterized sound, image, and movie data sets are an essential resource by which new research algorithms can be validated and compared in all fields of signal processing, including quantitative bioimaging, biometry, computer vision, and human-machine interaction. Availability of open, extensive, and well-characterized data sources allows comparing algorithm performance, prevents duplication of research efforts, helps ensure research results can be reproduced, and promotes sharing of computational data processing and analysis tools. Such multimedia data bases are most useful when the acquisition procedure, including all instruments are clearly characterized, permitting reproducibility of both the acquisition and processing procedures. While well-characterized data bases with static source and sensing devices are common (Idiap sustains an ongoing effort to curate and distribute such databases), it is less common to find data with moving sources or sensors, in 2D and 3D, and covering a wide array of motions, scales and modalities.This proposal aims at filling this gap by equipping the Idiap Research Institute with a dynamic data collection, processing, and distribution platform that will cover a broad range of applications and scales. The main focus will be the ability to reproducibly acquire data from moving objects (sources), with moving sensors, or with moving illumination, to cover a wide array of motion patterns or viewing directions in 2D and 3D (microscopy). Specifically, the platform will consist of a data storage and distribution platform (100+100TB), a high-performance computing core (6 GPU cores and 6 CPU cores) to assemble and serve the data, two robotic arms (\emph{UR3} and \emph{UR5} from Universal Robots) to collect highly dynamic sequences in a variety of controlled and reproducible motion patterns (moving sample, moving camera or illumination source), and a 3D fluorescence microscope (OpenSPIM). Beyond microscopy (a new research direction at Idiap), the platform will be scalable and sufficiently generic to support collecting data from modalities that are already well established at Idiap (sound, vision, biometry), yet it will enable new research direction by permitting sources and sensors to be dynamic, in some case a more realistic (yet precisely reproducible) representation of real-world application.
-