Projekt

Zurück zur Übersicht

Tracking in the Wild

Titel Englisch Tracking in the Wild
Gesuchsteller/in Fleuret François
Nummer 147693
Förderungsinstrument Sinergia
Forschungseinrichtung IDIAP Institut de Recherche
Hochschule Idiap Research Institute - IDIAP
Hauptdisziplin Informatik
Beginn/Ende 01.01.2014 - 31.12.2017
Bewilligter Betrag 994'500.00
Alle Daten anzeigen

Keywords (4)

Multi-camera tracking; Structure from Motion; Computer vision; Machine learning

Lay Summary (Französisch)

Lead
L'objectif de ce projet est d'améliorer les techniques de détection et de suivi automatique de personnes à l'aide de caméras de surveillance, en particulier en prenant en compte des aspects dynamiques complexes.L'automatisation de cette opération permettra d'assurer la sécurité dans des zones sensibles (manufactures, aéroports), tout en limitant au maximum l'accès au signal par des opérateurs lorsque cela n'est pas nécessaire, réduisant d'autant les problèmes de respect de la vie privée.
Lay summary
La détection et le suivi de personnes est un traitement automatique fondamental sous-jacent à de nombreuses applications de la vision par ordinateur.

La sécurité dans des manufactures, l'estimation de statistique dans des lieux publics, ou encore la surveillance de zones sensibles telles que les aéroports, reposent sur la capacité de systèmes informatiques à estimer à tout moment la présence de personnes dans des zones pré-définies, et à reconstruire des trajectoires complètes depuis leur arrivées sur la zone à surveiller jusqu'à leur départ.

Automatiser ces tâches permet en particulier de limiter l'accès systématique aux données par des opérateurs, ainsi que leur transmission ou leur stockage, améliorant d'autant le respect de la vie privée.

Ce projet vise à améliorer de tels systèmes, et est décomposé en trois parties:

La première partie s'intéresse à la reconstruction 3-D de l'environnement, en prenant en compte les aspects dynamiques. Ces derniers sont traités par la majorité des méthodes existantes comme des nuisances, alors que nous voulons les exploiter ici pour améliorer la reconstruction 3-D ainsi que le suivi développé dans les deux autres parties du projet.

La seconde a pour objectif de développer de nouvelles techniques d'apprentissage statistiques pour la détection jointe de personnes à l'aide de plusieurs caméras, et leur identification à partir d'un petit nombre d'exemples.

La troisième enfin concerne l'étude de modèles dynamiques riches, prenant en compte les interactions avec l'environnement, lui même modélisé de manière dynamique (voiture, portes, ballons, etc.)
Direktlink auf Lay Summary Letzte Aktualisierung: 21.11.2013

Verantw. Gesuchsteller/in und weitere Gesuchstellende

Mitarbeitende

Publikationen

Publikation
DARN: a Deep Adversial Residual Network for Intrinsic Image Decomposition
Lettry Louis, Vanhoey Kenneth (2018), DARN: a Deep Adversial Residual Network for Intrinsic Image Decomposition, in Winter Application Conference of Vision.
Kronecker Recurrent Units
Jose C., Cisse M., Fleuret F. (2018), Kronecker Recurrent Units, in Proceedings of the Workshop Track of the International Conference on Learning Representationsthe (IC.
SGAN: An Alternative Training of Generative Adversarial Networks
Chavdarova T., Fleuret F. (2018), SGAN: An Alternative Training of Generative Adversarial Networks, in Proceedings of the IEEE international conference on Computer Vision and Pattern Recognition (CVPR).
WILDTRACK: A Multi-camera HD Dataset for Dense Unscripted Pedestrian Detection
Chavdarova T., Baqué P., Bouquet S., Maksai A., Jose C., Bagautdinov T., Lettry L., Fua P., Van Gool L., Fleuret F. (2018), WILDTRACK: A Multi-camera HD Dataset for Dense Unscripted Pedestrian Detection, in Proceedings of the IEEE international conference on Computer Vision and Pattern Recognition (CVPR).
Deep Multi-Camera People Detection
Chavdarova T., Fleuret F. (2017), Deep Multi-Camera People Detection, in Proceedings of the IEEE International Conference on Machine Learning and Applications (ICMLA), 848-853, 848-853.
Non-Markovian Globally Consistent Multi-Object Tracking
Maksai A., Wang X., Fleuret F., Fua P. (2017), Non-Markovian Globally Consistent Multi-Object Tracking, in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2563-2573, 2563-2573.
Repeated Pattern Detection using CNN activations
Perdoch Michal, Vanhoey Kenneth, Gool Luc Van (2017), Repeated Pattern Detection using CNN activations, in Winter Application Conference of Vision.
Fast Optical Flow using Dense Inverse Search
Kroeger Till, Timofte Radu, Dai Dengxin, Gool Luc Van (2016), Fast Optical Flow using Dense Inverse Search, in Proceedings of the European Conference on Computer Vision, 9908, 471-488, 9908, 471-488.
Importance Sampling Tree for Large-scale Empirical Expectation
Canévet O., Jose C., Fleuret F. (2016), Importance Sampling Tree for Large-scale Empirical Expectation, in Proceedings of the International Conference on Machine Learning (ICML), 1454-1462, 1454-1462.
Importance Sampling Tree for Large-scale Empirical Expectation
Canévet O., Jose C., Fleuret F. (2016), Importance Sampling Tree for Large-scale Empirical Expectation, in Proceedings of the International Conference on Machine Learning (ICML), -, -.
Markov Chain Monte Carlo Cascade for Camera Network Calibration based on Unconstrained Pedestrian Tracklets
Lettry L., Dragon R., Gool L. Van (2016), Markov Chain Monte Carlo Cascade for Camera Network Calibration based on Unconstrained Pedestrian Tracklets, in Asian Conference of Computer Vision 2016, -, -.
Principled Parallel Mean-Field Inference for Discrete Random Fields
Baqué P., Bagautdinov T., Fleuret F., Fua P. (2016), Principled Parallel Mean-Field Inference for Discrete Random Fields, in Proceedings of the IEEE international conference on Computer Vision and Pattern Recognition (CVPR).
Scalable Metric Learning via Weighted Approximate Rank Component Analysis
Jose C., Fleuret F. (2016), Scalable Metric Learning via Weighted Approximate Rank Component Analysis, in Proceedings of the European Conference on Computer Vision (ECCV).
Scalable Metric Learning via Weighted Approximate Rank Component Analysis
Jose C., Fleuret F. (2016), Scalable Metric Learning via Weighted Approximate Rank Component Analysis, in Proceedings of the European Conference on Computer Vision (ECCV), 875-890, 875-890.
Kullback-Leibler Proximal Variational Inference
Mohammad Emtiyaz Khan, Pierre Baqué, François Fleuret, Pascal Fua (2015), Kullback-Leibler Proximal Variational Inference, in Proceedings of the international conference on Neural Information Processing Systems (NIPS).

Zusammenarbeit

Gruppe / Person Land
Formen der Zusammenarbeit
Technical University Graz Österreich (Europa)
- vertiefter/weiterführender Austausch von Ansätzen, Methoden oder Resultaten
University of Leuven Belgien (Europa)
- vertiefter/weiterführender Austausch von Ansätzen, Methoden oder Resultaten
- Publikation
- Austausch von Mitarbeitern
Universitat Politècnica de Catalunya Spanien (Europa)
- vertiefter/weiterführender Austausch von Ansätzen, Methoden oder Resultaten
- Publikation
- Austausch von Mitarbeitern
Technical University of Munich Deutschland (Europa)
- vertiefter/weiterführender Austausch von Ansätzen, Methoden oder Resultaten

Wissenschaftliche Veranstaltungen



Selber organisiert

Titel Datum Ort
Workshop on Deep Learning, Tools and Methods. 04.07.2016 Idiap, Martigny, Schweiz

Verbundene Projekte

Nummer Titel Start Förderungsinstrument
159248 Motion Models for Monocular People Tracking 01.09.2015 Projektförderung (Abt. I-III)
172500 Modeling People and their Clothes in Crowded Scenes 01.01.2018 Projektförderung (Abt. I-III)
169112 Importance sampling for large-scale unsupervised learning (ISUL) 01.03.2017 Projektförderung (Abt. I-III)
164030 Multi-Camera System for Behavioral Analysis 01.01.2016 R'EQUIP
140912 Object Detection with Active Sample Harvesting (DASH ) 01.10.2012 Projektförderung (Abt. I-III)
132167 Aerial Crowds: Populating Mixed Reality Cities 01.02.2011 Sinergia
131549 Advanced Learning for Tracking and Detection in Medical Workflow Analysis. 01.04.2012 Projektförderung (Abt. I-III)
144318 Motion Models for Monocular People Tracking 01.09.2013 Projektförderung (Abt. I-III)

Abstract

People tracking is central to many applications ranging from surveillance in complex urban environments to behavioral analysis in cluttered work spaces. However, in spite of years of sustained research, existing approaches can still only operate successfully in constrained environments, such as a sport arena, or for restricted subsets of human activities, such as walking along a city street. The goal of this project is to dramatically broaden the scope of current methods so that the resulting algorithms can be ``taken into the wild,'' that is, to far more unconstrained and generic settings.To this end, we will build on the multi-camera multi-target approach we have developed jointly over many years (Fleuret et al., 2008; Berclaz et al., 2011). Recently, we have focused on tracking basketball and soccer players (Ben Shitrit et al., 2011, 2012) and outperform state-of-the-art approaches. However, this is only true in controlled environments, and our current system cannot operate in cluttered real-world public spaces for the following three reasons: First, it requires multiple cameras, each carefully calibrated, and assumes a planar area of interest in which the only moving things are the people. Second, it relies on background subtraction, which is sensitive to global illumination changes, and on appearance models that have to be learned prior to using the system. Third, it cannot leverage sophisticated motion models either at the individual target level or at the group level.The objective of WildTrack is to eliminate these weaknesses, which will require the joint expertise of all three partners. The research will be organized around a joint benchmarking platform and will be decomposed into the following three sub-projects.Sub-project 1 - Environment Modeling and Camera Calibration:Our existing tracking system relies on camera calibration, a planar ground, and entrances and exits restricted to the edges of the area of interest. It also ignores potential occluders, such as pillars that limit the field of views of some cameras. This sub-project will rely on Structure-from-Motion (SfM) techniques combined with object class detections and tracking results to produce a more refined 3D model of the environment. This includes breaking loose of planarity restrictions on the ground geometry, obtaining knowledge about typical trajectories, as well as about probable sources and sinks for the moving objects. We will consider both cases where traditional uncalibrated SfM can be applied---sufficiently many cameras with enough fields-of-view overlap---and cases where that is not true and information about object classes and probable trajectories is all the more important to compensate for the failure of normal SfM.Sub-project 2 - Large-Scale Learning for Detection and Recognition:Our current implementation relies on background-subtraction to detect humans and on crude color-based appearance models to disambiguate difficult tracking situations, neither of which is robust to changes in imaging conditions. This sub-project will tackle this weakness by learning from large training sets. We will first collect videos with multiple cameras and keep trajectories for which we have high prediction confidence. Data gathered along these trajectories will be used to train predictors to detect moving objects visible in a limited number of views, potentially corrupted by noise. It will also allow the use of transfer learning that will make it possible to learn someone's appearance from a handful of exemplar images.Sub-project 3 -- Convex High-dimension Tracking:At present, we characterize people solely by their 2D ground positions, thus ignoring their 3D poses and interactions with each other and inanimate objects. This is sufficient when tracking pedestrians whose range of motion is small but is limiting when dealing with more complex behaviors, such as those of people sitting, standing, or reclining in the course of their daily lives. Removing these limitations will require performing our multi-target tracking in much higher dimensional state-spaces than the ones we have worked with so far, and will be the focus of this sub-project.These objectives require in-depth expertise in many core areas of Computer Vision and Machine Learning, which the three research groups involved in this project possess collectively. As a result, it will improve the state-of-art and yield a people-tracking approach that can truly be deployed in the wild.
-