Back to overview

Synergistic Approach to Capturing and Exploiting Microscopy Images

Applicant Fua Pascal
Number 177237
Funding scheme Sinergia
Research institution Laboratoire de vision par ordinateur EPFL - IC - ISIM - CVLAB
Institution of higher education EPF Lausanne - EPFL
Main discipline Interdisciplinary
Start/End 01.09.2018 - 31.01.2023
Approved amount 2'143'144.00
Show all

All Disciplines (5)

Material Sciences
Information Technology
Neurophysiology and Brain Research

Keywords (6)

Stereography; Light Microscopy; Dislocations; Electron Microscopy; Brain Connectivity; Delineation

Lay Summary (French)

À mesure que les microscopes optiques et électroniques deviennent de plus en plus puissants, ils peuvent produire des images à haute résolution en quantités si importantes que le goulot d'étranglement devient le traitement et l’interprétation de ces image. Malheureusement, les techniques d'analyse d'image actuelles reposent largement sur l'intervention manuelle. Ceci réduit considérablement la quantité de données pouvant être traitées et limite la recherche à des preuves anecdotiques plutôt que statistiques. L'objectif de ce projet est de remédier à ce problème.
Lay summary

L'objectif de ce projet est de fournir une approche radicalement nouvelle et intégrée où les images sont acquises de manière à ce que les outils de reconstruction automatisés soient les plus efficaces. À cette fin, nous voulons combiner notre expertise commune en matière d’imagerie, de vision par ordinateur, et de représentation des structures pour nous assurer que les algorithmes que nous développerons fourniront les meilleurs résultats possibles. Nous travaillerons sur un large éventail de structures de complexité croissante. Nous avons choisi de travailler sur des structures venant de différents domaines car elles partagent des propriétés topologiques communes, ce qui signifie que les mêmes types de méthodes et d'algorithmes peuvent être utilisés pour toutes. Cependant, elles présentent également des différences: Elles vont de structures purement linéaire, comme les dislocations dans les cristaux, les structures neurales et les fibrilles amyloïdes, aux structures partiellement linéaires et partiellement membraneuses, telles que le réticulum endoplasmique, et enfin aux structures purement membraneuses. Certaines d’entre elles peuvent être imagées pour produire des volumes intrinsèquement 3D, tandis que d’autres seront projetées dans des images 2D, faisant que la récupération de leur structure 3D nécessitera une reconstruction stéréo. 

Les approches que proposerons seront suffisamment génériques pour les traiter toutes, ce qui garantira que les résultats du projet auront un impact au-delà de la résolution des problèmes de reconstruction spécifiques que nous ciblons. De plus,  transformerons nos algorithmes en outils logiciels fiables et conviviaux que les scientifiques pourront utiliser dans leur travail quotidien pour modéliser rapidement des volumes de données volumineux et complexes et accroître ainsi considérablement leur productivité. À leur tour, ils développeront des protocoles d'imagerie optimisés pour l'analyse d'image. Nous rendrons ensuite ces outils et protocoles publiquement accessibles afin que les chercheurs de nombreux domaines puissent en bénéficier.

Direct link to Lay Summary Last update: 30.08.2018

Responsible applicant and co-applicants


Project partner


Anatomically and functionally distinct thalamocortical inputs to primary and secondary mouse whisker somatosensory cortices
El-Boustani Sami, Sermet B. Semihcan, Foustoukos Georgios, Oram Tess B., Yizhar Ofer, Petersen Carl C. H. (2020), Anatomically and functionally distinct thalamocortical inputs to primary and secondary mouse whisker somatosensory cortices, in Nature Communications, 11(1), 3342-3342.
Sensorimotor processing in the rodent barrel cortex
Petersen Carl (2019), Sensorimotor processing in the rodent barrel cortex, in Nature reviews neuroscience, 533-546.
A Topological “Reading” Lesson: Classification of MNIST using TDA
Garin Adélie, Tauzin Guillaume, A Topological “Reading” Lesson: Classification of MNIST using TDA, in International Conference On Machine Learning And Applications, IEEE, Boca Raton, USA.
TopoAL: An Adversarial Learning Approach for Topology-Aware Road Segmentation
VasuSubeesh, KozinskiMateusz, CitraroLeonardo, TopoAL: An Adversarial Learning Approach for Topology-Aware Road Segmentation, in European Conference on Computer Vision, ECCV, Online.

Associated projects

Number Title Start Funding scheme
166107 Domain Adaptation for Microscopy Imaging 01.08.2016 Project funding
204231 Modeling People and their Clothes in Crowded Scenes 01.01.2022 Project funding
182533 Operads, Calculus and Homotopy theory methods in Topology 01.01.2019 Project funding
182010 Neural circuits for goal-directed sensorimotor transformation 01.04.2019 Project funding
170795 High resolution fluorescence imaging across large volumes of intact organs 01.12.2016 R'EQUIP


As light and electron microscopes become ever more powerful, they can produce high-resolution imagery in such large quantities that the bottleneck becomes processing and interpretation of their output. Unfortunately, current image analysis techniques rely heavily on manual intervention and feedback, which severely limits the amount of data that can be processed and restricts research to anecdotal rather than statistical evidence. Furthermore, since human annotators may have different biases and opinions about what constitutes a significant feature, their involvement introduces an undesirable element of unpredictability in the final interpretation. In the worst case, important structures that cannot easily be discerned by human observers may be overlooked. Consequently, targeted data acquisition taking into account the whole chain of data treatments as well as sample specificities is required to take full advantage of the power of cutting-edge instruments.The goal of this project is to provide a radically new, integrated, approach where images are acquired in such a way that the automated reconstruction tools are most effective. To this end, we want to combine our joint expertise in imaging, computer vision, and structure-representation to ensure the algorithms we will develop deliver the best possible results. We will work on a wide range of scientifically relevant structures of increasing complexity. We chose to work on all these structures from different domains because they share common topological properties, which means the same kinds of methods and algorithms can be used on all of them. However, they also exhibit differences: They range from purely linear, such as dislocations, neural structures, and amyloid fibrils, to structures that are partly linear and partly membranous, such as the endoplasmic reticulum, and finally to purely membranous ones. Some of them can be imaged to produce stacks that are inherently 3D, while others will be seen in multiple 2D images, so that recovering their 3D structure will involve stereo reconstruction. The approaches we propose will therefore have to be generic enough to handle them all, which will guarantee that the output of the project will have an impact beyond solving the specific reconstruction problems we target. We will systematically make sure that the researchers who analyze the images provide continuous feedback to those who acquire them so that they can jointly and continuously improve the imaging protocols to guarantee optimal performance for the algorithms. This is a departure from traditional practice, which involves first imaging the data and then having the algorithms deal with the result, no matter how well or ill-suited it is for automated processing. Both because of this and because we will integrate into our algorithms powerful new Machine Learning and Graph Optimization techniques, they will become far more effective than before and allow us to analyze unprecedented amounts of image data. This will enable us to reconstruct brain-wide axonal arborizations of hundreds of individual neurons, which will constitute a landmark achievement for neuroscience. It will also pave the way towards automated guidance of Material Scientists through the Transmission Electron Microscopy (TEM) image acquisition process and deliver much faster analysis of 3D morphologies. This will unleash the full power of modern TEMs and represent a crucial step toward real-time 3D TEM, a holy grail not only for Material Science but also for Bioengineering and Life Science.Finally, we will turn our algorithms into reliable, user-friendly software tools that material scientists and neuroscientists can use in their daily work to quickly model large and complicated data volumes and thereby tremendously increase their productivity. In turn, they will develop imaging protocols that are optimized for image analysis. We will then make both the tools and protocols publicly available so that researchers in many different fields can benefit from them.