Project

Back to overview

Modeling People and their Clothes in Crowded Scenes

English title Modeling People and their Clothes in Crowded Scenes
Applicant Fua Pascal
Number 172500
Funding scheme Project funding (Div. I-III)
Research institution Laboratoire de vision par ordinateur EPFL - IC - ISIM - CVLAB
Institution of higher education EPF Lausanne - EPFL
Main discipline Information Technology
Start/End 01.01.2018 - 31.12.2021
Approved amount 789'037.00
Show all

Keywords (4)

Computer Vision ; People Tracking; Surface Modeling; Pose Estimation

Lay Summary (French)

Lead
La modélisation du corps humain et de ses mouvements reste difficile pour plusieurs raisons: Les humains ont une géométrie complexe. Leurs vêtements, en se déformant, rendent l’analyse du mouvement plus difficile, et certaines parties du corps sont souvent invisibles. C’est la problématique sur laquelle nous entendons travailler. Les applications potentielles en sont la formation sportive, la surveillance, le divertissement, et le suivi médical.
Lay summary

De nombreux progrès ont été réalisés récemment, en grande partie grâce à l'avènement de techniques puissantes d'apprentissage statistique Cependant, la tâche reste difficile en raison des nombreuses ambiguïtés inhérentes à la reconstruction 3D à partir d’images 2D. Dans nos travaux antérieurs, nous avons abordé ces difficultés dans le contexte de l'estimation de la pose tridimensionnelle du corps et de la reconstruction de surfaces.

Bien que ces approches soient efficaces pour des personnes ou surfaces individuelles, elles ne tiennent compte ni des interactions entre personnes alors que leur mouvement est souvent corrélé ni de celles entre personnes et leurs vêtements, qui sont des surfaces déformables dont le mouvement est fonction de celui qui les porte. Le but de ce projet est donc de combiner nos approches antérieures et de prendre en compte les interactions pour que nous puissions enfin modéliser des groupes de personnes habillées normalement.

Direct link to Lay Summary Last update: 29.11.2017

Responsible applicant and co-applicants

Employees

Publications

Publication
ActiveMoCap: Optimized Viewpoint Selection for Active Human Motion Capture
Kiciroglu Sena, Rhodin Helge, Sinha Sudipta, Salzmann Mathieu, Fua Pascal (2020), ActiveMoCap: Optimized Viewpoint Selection for Active Human Motion Capture, in CVPR, IEEE, Online.
Estimating People Flows to Better Count Them in Crowded Scenes
Liu Weizhe, Salzmann Mathieu (2020), Estimating People Flows to Better Count Them in Crowded Scenes, in ECCV, EUROPEAN CONFERENCE ON COMPUTER VISION, Online.
MeshSDF: Differentiable Iso-Surface Extraction
Remelli Edoardo, Lukoianov Artem, Richter Stephan, Guillard Benoit, Bagautdinov Timur, Baque Pierre, Fua Pascal (2020), MeshSDF: Differentiable Iso-Surface Extraction, in NeurIPS, Neural Information Processing Systems Foundation, Online.
Shape Reconstruction by Learning
BednarikJan, ParasharShaifali, GundogduErhan, SalzmannMathieu (2020), Shape Reconstruction by Learning, in CVPR, IEEE, Online.
UCLID-Net: Single View Reconstruction in Object Space
Guillard Benoit, Remelli Edoardo, Fua Pascal (2020), UCLID-Net: Single View Reconstruction in Object Space, in NeurIPS, Neural Information Processing Systems Foundation, Online.
Eliminating Exposure Bias and Metric Mismatch in Multiple Object Tracking
Maksai Andrii, Fua Pascal (2019), Eliminating Exposure Bias and Metric Mismatch in Multiple Object Tracking, in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USAIEEE, Long Beach, CA, USA.

Associated projects

Number Title Start Funding scheme
166107 Domain Adaptation for Microscopy Imaging 01.08.2016 Project funding (Div. I-III)
147693 Tracking in the Wild 01.01.2014 Sinergia
163461 Modeling Deformable 3-D Surfaces from Video 01.09.2016 Project funding (Div. I-III)
159248 Motion Models for Monocular People Tracking 01.09.2015 Project funding (Div. I-III)

Abstract

Capturing people's 3D shape and motion has always been central to Computer Vision. Today, there is great interest in doing so solely by analyzing video sequences, both because cameras are becoming ever cheaper and more prevalent and because there are so many potential applications. They include athletic training, surveillance, entertainment, and fashion design among others.Much progress has been made recently, in large part thanks to the advent of powerful Machine Learning techniques. However, the task remains challenging because of the many ambiguities inherent to monocular 3D reconstruction, which stem from the loss of depth information resulting from the projection from 3D to 2D. In earlier work, we have addressed these difficulties in the contexts of body pose estimation and surface reconstruction.While both of these approaches are effective on individual people or surfaces, they completely ignore the interactions both between people whose motion are often correlated as well as between people and their clothes, which are deformable surfaces whose motion is driven by that of the person wearing them. The goal of this project will therefore be to bring these two strands of research together so that we can ultimately model multiple clothed people interacting with each other, which is still beyond the state of the art.
-