Back to overview

Very Large Sets of Heuristics for Scene Interpretation (VELASH )

Applicant Fleuret François
Number 124822
Funding scheme Project funding (Div. I-III)
Research institution IDIAP Institut de Recherche
Institution of higher education Idiap Research Institute - IDIAP
Main discipline Information Technology
Start/End 01.09.2009 - 31.08.2012
Approved amount 155'526.00
Show all

Keywords (7)

Machine learning; Pattern recognition; Artificial vision; Object recognition; Object detection; Scene interpretation; Complex prior knowledge

Lay Summary (English)

Lay summary
Object detection aims at automatically identifying and localizing classes of objects in still images. Software able to perform such a task is central in fields as diverse as biometric authentication, automatic surveillance or robot navigation.Most of the state-of-the-art object detection techniques are based on statistical learning. They use large sets of examples to automatically infer the regularity and specificity of a class of object to characterize it visually. The center issue experts have to deal with in such a context is invariance. They have to design adequate representations of the image based on a prior knowledge of the problem, so that the statistical learning itself can focus on unknown randomness.The standard example of such a processing is edge detection. By feeding the machine learning with an image of edges instead of the original image, one removes the need for learning invariance to illumination, which is not present in the signal anymore. Such low-level "features" are known to exist in the visual processing of animals.While fundamental, the complexity of such pre-processing steps has remained pretty low. Most of the effort has been focused on improving methods based on a restricted family of image descriptors, instead of trying to increase the versatility and richness of the feature set.The goal of this project is to investigate a new approach to object detection and machine learning in general by combining state of the art learning methods with very rich families of feature extractors. Instead of limiting ourselves to one representation of the image, we will study how to combine efficiently different families of features, and how to help experts design them.The motivation behind this project is twofold. From a practical stance we are trying to leverage the robustness resulting from the combination of a large number of modalities designed by different experts. From a more fundamental perspective, we hope to reduce the gap between artificial and biological cognition by reducing the burden on the learning part.
Direct link to Lay Summary Last update: 21.02.2013

Responsible applicant and co-applicants


Name Institute


Adaptive Sampling for Large Scale Boosting
Dubout Charles, Fleuret Francois (2014), Adaptive Sampling for Large Scale Boosting, in Journal of Machine Learning Research, 15, 1431-1453.
Accelerated Training of Linear Object Detectors
Dubout Charles, Fleuret Francois (2013), Accelerated Training of Linear Object Detectors, in Proceedings of theIEEE international conference on Computer Vision and Pattern Recognition Workshops, Portland, OregonIEEE, New-York.
Deformable Part Models with Individual Part Scaling
Dubout Charles, Fleuret Francois (2013), Deformable Part Models with Individual Part Scaling, in Proceedings of the British Machine Vision Conference, BMVA, England.
Exact Acceleration of Linear Object Detectors
Dubout Charles, Fleuret Francois (2012), Exact Acceleration of Linear Object Detectors, in Proceedings of the European Conference on Computer Vision, FirenzeSpringer, Berlin Heidelberg.
Boosting with Maximum Adaptive Sampling
Dubout C., Fleuret F. (2011), Boosting with Maximum Adaptive Sampling, in Proceedings of the international conference on Neural Information Processing Systems, Granada, SpainProceedings of the Neural Information Processing Systems Conference (NIPS), n.a..
Comparing machines and humans on a visual categorization test
Fleuret F., Li T., Dubout C., Wampler E. K., Yantis S., Geman D. (2011), Comparing machines and humans on a visual categorization test, in Proceedings of the National Academy of Sciences (PNAS), 108(43), 17621-1762.
Tasting Families of Features for Image Classification
Dubout C., Fleuret F. (2011), Tasting Families of Features for Image Classification, in Proceedings of the IEEE International Conference on Computer Vision, Barcelona, SpainProceedings of the IEEE International Conference on Computer Vision (ICCV), n.a..
The MASH project
Fleuret F., Abbet P., Dubout C., Lefakis L., The MASH project, in Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge , Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge , n.a..


Title Year
Idiap PhD Student Paper Award 2011


Title Date Number Inventor Owner
Object detection method, object detector and object detection computer program 21.09.2012 US13/624375

Associated projects

Number Title Start Funding scheme
140912 Object Detection with Active Sample Harvesting (DASH ) 01.10.2012 Project funding (Div. I-III)
140941 Very Large Sets of Heuristics for Scene Interpretation (VELASH) 01.09.2012 Project funding (Div. I-III)


The goal of the research in machine learning has been so far to avoid the necessity for a detailed hand-designed prior knowledge. This proposal takes the opposite stance and aims precisely at combining a very large number of heuristics in a statistical framework. It is an attempt at bootstrapping an academic interest for the development of structurally complex priors for applied machine learning.We define an heuristic to be any feature extractor, an algorithm processing the available raw signal to produce values relevant to the problem at hand. This purposely general definition encompasses techniques spanning from simple rules to symbolic modeling or unsupervised locally trained predictors. We assume that high performance can only be achieved by combining thousands of such hand-designed modules, and propose to develop these heuristics in an open and web-based collaborative framework similar to the successful development process of open-source software and collaborative encyclopedia. As many contributors will be involved, the resulting system will benefit from the completeness andredundancy of as many idiosyncratic viewpoints.Although of potential interest to many applications, we study this approach for the problem of scene interpretation. Given an image with every day objects and furniture or an outdoor landscape, the goal is to detect and identify as many objects as possible. Solving this task requires to design aclassifier able to recognize the identity of an isolated object. We propose to combine a multitude of feature extractors capturing different aspects of the image to create this multi-class predictor.The research to be tackle in this project can be structured in two main axis:1. The development of a machine learning technique to aggregate the heuristic responses for object classifcation. We will have to handle the classical difficulties arising in such situation of very large dimension, with the additional difficulty of dealing with an highly heterogeneous feature space. Our initial choice will be forests of decision trees, which have many properties, statistical and algorithmic, desirable for that project.2. The study of feedback tools to help and motivate the design of heuristics. Such tools will have to produce a ranking of the heuristics to provide the contributors with both a good estimate of their own progress, a good perception of their overall ranking in the community of contributors, and an assistance in figuring what are the main weakness of the current set of heuristics.Performance will be measured on real images such as the Caltech 256 or the VOC dataset to allow for comparison with state-of-the-art techniques, and on computer-generated images, which can be produced along an exhaustive labeling with very limited resources.The approach described here aims at creating a novel research field of "heuristic mining". Instead of exploiting a fixed set of homogeneous features, trying to cope with ist incompleteness, research will be focused on developing strategies to motivate the development of very large feature sets.