Project

Back to overview

Modeling Deformable 3-D Surfaces from Video

English title Modeling Deformable 3-D Surfaces from Video
Applicant Fua Pascal
Number 126524
Funding scheme Project funding (Div. I-III)
Research institution Laboratoire de vision par ordinateur EPFL - IC - ISIM - CVLAB
Institution of higher education EPF Lausanne - EPFL
Main discipline Information Technology
Start/End 01.10.2009 - 30.09.2011
Approved amount 108'880.00
Show all

Keywords (3)

Surface Modeling; Video; Video-Based Surface Modeling

Lay Summary (English)

Lead
Lay summary
Being able to recover the 3D shape of deformable surfaces using a single camera would make it possible to field reconstruction systems that run on widely available hardware. However, because many different 3D shapes can have virtually the same projection, such monocular shape recovery is inherently ambiguous. In an earlier phase of this project, we showed that simply constraining the distances between selected surface points to remain constant is enough to recover 3D shape from a single input image, provided that point correspondences can be established with a reference image in which the shape is known. Furthermore, the shape-recovery problem can be formulated as a convex problem that can be solved using standard mathematical routines and does not require an initial shape estimate.In the current phase of this project, we intend to extend our approach in three complementary directions:- We will remove the requirement for a reference configuration by developing algorithms to build it from short image sequences in which we will detect and track points that move consistently. We will use them to reconstruct small patches that we will then assemble into complete surfaces.- We will increase the reliability of our point matching scheme by performing it in conjunction with 3--D shape estimation to take advantage of the fact that correspondences ought to be geometrically consistent.- We will increase the quality of our shape reconstruction scheme by bringing in additional information, such as shading information, to disambiguate difficult cases that arise when it is difficult to differentiate a concavity from a convexity based on correspondences alone.Bringing together these three strands of research will result in 3--D surface reconstruction algorithms that are more robust, more accurate, and much easier to deploy in real-world applications.
Direct link to Lay Summary Last update: 21.02.2013

Responsible applicant and co-applicants

Employees

Name Institute

Publications

Publication
Linear Local Models for Monocular Reconstruction of Deformable Surfaces
Salzmann M., Fua P. (2011), Linear Local Models for Monocular Reconstruction of Deformable Surfaces, in IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5).
Template-Free Monocular Reconstruction of Deformable Surfaces
Varol A., Salzmann M., Tola E., Fua P. (2009), Template-Free Monocular Reconstruction of Deformable Surfaces, IEEE Xplore, Kyoto.
Monocular 3D Reconstruction of Sparsely Textured Surfaces
Varol A., Shaji A., Salzmann M., Fua P., Monocular 3D Reconstruction of Sparsely Textured Surfaces, in IEEE Transactions on Pattern Analysis and Machine Intelligence.

Associated projects

Number Title Start Funding scheme
113279 Modeling Deformable 3-D Surfaces from Video 01.10.2007 Project funding (Div. I-III)
137525 Modeling Deformable 3-D Surfaces from Video 01.01.2012 Project funding (Div. I-III)
153121 Modeling Deformable 3-D Surfaces from Video 01.04.2014 Project funding (Div. I-III)

Abstract

While reconstruction and tracking of rigid and articulated objects from video have been widely studied, modeling 3--D deformable surfaces remains a challenging problem, especially when using a single camera. The problem would be totally under-constrained without an appropriate deformation model, that is, one flexible enough to account for all possible configurations but controlled by sufficiently few parameters for effective optimization.Physics-based models have been extensively investigated as a potential answer to this problem. They have been shown to be excellent at fitting noisy image data and handling highly deformable objects. However, it is important to realize that many of these models rely on oversimplified, often quadratic, regularization terms to couple their degrees of freedom. These terms do not always accurately represent the physics of what is known as large deformations in the Mechanical Engineering community. This requires higher order terms to represent highly non-linear stresses and strains.In this project, we will develop models designed to work under more challenging conditions by accounting more accurately than current approaches for the material properties of surfaces, while striving to retain the simplicity of the original deformable models. This will increase the robustness of the algorithms that rely on such models, without significantly increasing their computational complexity.
-