Project

Back to overview

Modeling People and their Clothes in Crowded Scenes

English title Modeling People and their Clothes in Crowded Scenes
Applicant Fua Pascal
Number 204231
Funding scheme Project funding
Research institution Laboratoire de vision par ordinateur EPFL - IC - ISIM - CVLAB
Institution of higher education EPF Lausanne - EPFL
Main discipline Information Technology
Start/End 01.01.2022 - 31.12.2025
Approved amount 995'404.00
Show all

Keywords (4)

People Tracking; Computer Vision ; Surface Modeling; Pose Estimation

Lay Summary (French)

Lead
La modélisation du corps humain et de ses mouvements à partir d’images vidéo est difficile pour plusieurs raisons : Les humains ont une géométrie complexe. Leurs vêtements, en se déformant, rendent l’analyse du mouvement plus difficile, et certaines parties du corps sont souvent invisibles. C’est une problématique importante avec de nombreuses applications potentielles dans les domaines de la formation sportive, de la surveillance, du divertissement, et du suivi médical, entre autres.
Lay summary

De nombreux progrès ont été réalisés récemment, en grande partie grâce à l'avènement de techniques puissantes d'apprentissage statistique Cependant, la tâche reste difficile en raison des nombreuses ambiguïtés inhérentes à la reconstruction 3D à partir d’images 2D.

Dans nos travaux antérieurs, nous avons abordé ces difficultés dans le contexte de l'estimation de la pose de personnes portant des vêtements relativement moulants et qui se déforment comme le corps. Dans ce nouveau projet, nous aborderons la problématique des personnes qui portent des vêtements plus amples et qui ont leur mouvement propre. Cela nous permettra de développer des techniques qui peuvent opérer dans des environnements de tous les jours où les gens sont habillés de façon normale.

Direct link to Lay Summary Last update: 07.12.2021

Responsible applicant and co-applicants

Employees

Associated projects

Number Title Start Funding scheme
177237 Synergistic Approach to Capturing and Exploiting Microscopy Images 01.09.2018 Sinergia
172500 Modeling People and their Clothes in Crowded Scenes 01.01.2018 Project funding

Abstract

Current algorithms for modeling clothed people mainly fall into one of two categories, those that model the external surface of the clothed body without modeling the body itself and those that model the body while ignoring the clothes. The former have been shown to work remarkably well on very clean images in which the person can be precisely outlined but rarely account for poor lighting or occlusions that are frequent in the real world. The latter are more robust but do not fully exploit the information the shape of the clothes provides about that of the body. Furthermore, in neither case are many provisions made for the fact that there may be many people in the scene and that they may occlude each other. As a result, whereas modeling people wearing tight-fitting clothes is fast becoming a mature field, handling subjects wearing looser garments remains an open problem when it is to be done in everyday settings where people may hide each other, precise outlines are hard to estimate, and shadows often complicate matters. In this project, we will handle this more difficult task of modeling people and their potentially loose fitting clothing from video. This will include modeling individuals as well as multiple people interacting with each other. In the process, we will also develop new approaches to representing complex surfaces whose topology can change and to modeling their interactions. We believe that this will have an impact beyond the specific topic of people and their clothes, for example in the field of Computer Aided Design where such models are crucial. Finally, by deploying our algorithms in the real-world setting of drones filming people, we will prove that the potential of our technique goes beyond lab environments, which eventually will lead to technology transfer from Academia to Industry.
-