Project

3D Reconstruction ; Deformable Models ; Surface Modeling; Computer Vision

Lead
Pouvoir retrouver la forme 3D de surfaces déformables en n’utilisant qu’une seule caméra permettrait de le faire non seulement avec toutes celles qui équipent maintenant tous les téléphones portables et tablettes mobiles mais aussi dans des contextes plus spécialisés comme celui de la chirurgie endoscopique. Malheureusement c’est un problème difficile parce que beaucoup de formes 3D peuvent se projeter dans les images de manière très similaire. Dans ce projets, nous nous attachons à résoudre les ambiguïtés qui en résultent.
Lay summary
Pour résoudre le problème de la reconstruction monoculaire de surfaces déformables, nous avons développé des approches qui reposent sur ??l'établissement de correspondances entre dans une image d'entrée dans laquelle la forme 3D doit être retrouvée et une image de référence dans laquelle elle est connue. Nos algorithmes peuvent désormais gérer de manière fiable la déformation des surfaces 3D, mais ne prennent  pas en compte leur environnement. Il s'agit d'une limitation sévère parce que, dans le monde réel  les objets et les surfaces n’existent pas indépendamment les uns des autres. Au contraire, ils interagissent et la modélisation de ces interactions est essentielle pour la reconstruction précise et la compréhension des phénomènes physiques mis en jeu. Ce sont ces interactions que nous allons étudier dans ce projet. Nous nous concentrerons sur la modélisation des zones de contact, par exemple dans le cas d’une balle frappée par une raquette de tennis, une batte de baseball, ou un club de golf et aussi dans celui d’un organe déformé par des instruments chirurgicaux au cours d’une opération.

Direct link to Lay Summary

Last update: 28.03.2014

Title	Year

Number	Title	Start	Funding scheme

Being able to recover the 3D shape of deformable surfaces using a single camera will make it possible to field reconstruction systems that only require ordinary passive cameras, such as those that now equip most mobile devices. It will also make 3D shape recovery possible in more specialized contexts, such as when performing endoscopic surgery or using a fast camera to capture the deformations of a rapidly moving object. However, because many different 3D shapes can have virtually the same projection, such monocular shape recovery is inherently ambiguous.The solutions that have been proposed over the years mainly fall into two classes: Those that involve physics-inspired models and those that rely on a non-rigid structure-from-motion approach. The former solutions often entail designing complex objective functions and require hard-to-obtain knowledge about the precise material properties of the target surfaces. The latter depend on points being reliably tracked in image sequences and are only effective for relatively small deformations.To overcome these limitations, we have developed approaches that rely on establishing correspondences between in an input image in which the 3D shape is to be recovered and a reference image in which the 3D shape is known. We showed that 3D shape recovery under those conditions could be formulated as an under-constrained linear problem. Furthermore, either introducing inextensibility constraints or using control points to reduce the dimensionality of the problem turns it into a well-posed problem, which can be solved either in closed-form or using convex optimization.Our algorithms can now reliably handle deforming 3D surfaces but do not take their environment into account. This is a severe limitation because, in the real world, objects and surfaces do no exist in isolation. Instead, they interact with each other and modeling these interactions is key to accurate reconstruction and understanding of the physical phenomena at play. Examples include balls being hit by rackets, bats, and clubs at ballgames and organs being prodded by surgical tools during operations.This is the topic we intend to address in the continuation of this project and we will focus on the following two issues.- Modeling contact areas: When two objects come into contact, the contact area is hidden from the camera but should nevertheless be modeled properly to impose the right constraints on the 3D reconstruction and increase accuracy. We will therefore introduce consistency constraints that guarantee physically possible behavior of the contact area.- Explicitly modeling folds: Around contact points, surface texture often becomes difficult to exploit but shape information can be obtained from the surface folds that usually appear. We will therefore look into approaches to automatically detecting these folds and using them to impose appropriate differential constraints on the reconstructed surfaces.This will result in 3--D surface reconstruction algorithms that are more robust, more accurate, and can truly be deployed in real-world applications involving objects that interact with each other.