Project

Back to overview

BLIND 3D FACE DEBLURRING

English title BLIND 3D FACE DEBLURRING
Applicant Favaro Paolo
Number 165845
Funding scheme Project funding (Div. I-III)
Research institution Institut für Informatik Universität Bern
Institution of higher education University of Berne - BE
Main discipline Information Technology
Start/End 01.01.2017 - 30.09.2020
Approved amount 177'463.00
Show all

All Disciplines (2)

Discipline
Information Technology
Mathematics

Keywords (5)

fast search; 3D motion; 3D face; discriminative methods; image deblurring

Lay Summary (Italian)

Lead
Elaborazione digitale di immagini per ricostruire facce sfocate
Lay summary

Un'esperienza comune a molti genitori e' la difficolta' di fare foto ai loro figli nei loro momenti piu' importanti. Spesso queste foto sono rovinate da sfocamento dovuto al movimento, il cosiddetto motion blur. In particolare, i dettagli che vengono compromessi di piu' sono quelli relativi al viso. Purtroppo il motion blur non e' solo causato dal movimento inavvertito del genitore ma anche e soprattutto dal movimento del soggetto della foto, che spesso non può essere controllato. 

Per contrastare questo problema, proponiamo dei metodi di elaborazione digitale per la ricostruzione di immagini degradate dal motion blur. Vista l'estrema complessità del problema e la scarsezza dei dati, l'approccio generale e' di consentire l'uso di interazione con l'utente e di usare modelli di facce. Considereremo anche l'estensione dei nostri metodi ad altri soggetti come l'intero corpo, veicoli, animali, o singole parti del corpo (per esempio mani, braccia, gambe) e gli oggetti con i quali queste interagiscono.

Direct link to Lay Summary Last update: 09.08.2016

Responsible applicant and co-applicants

Employees

Name Institute

Associated projects

Number Title Start Funding scheme
153324 LIGHT FIELD MOTION AND TURBULENCE DEBLURRING 01.01.2015 Project funding (Div. I-III)

Abstract

In this proposal we describe a research plan to restore images depicting blurred faces. We investigate methods based on 3D models of faces and to cope with the task complexity and ambiguities we allow user interaction. We focus on faces because of their societal importance, but we expect that the proposed method could be applied to similar image deblurring tasks, which would be addressed in future work. Possible extensions are, for example, the deblurring of a whole person, vehicles, animals, or just body parts such as hands and legs (for example, together with baseball bats, a soccer ball, and similar sports equipment).If we accept that there is no easy way to capture blur-proof pictures, then one way to see their sharp version is to undo the blur distortion computationally. Unfortunately, current deblurring methods make very restrictive assumptions. For example, they assume that the scene is a single plane or that blur varies smoothly across the image domain. While these methods work for many interesting cases, they do not apply to our scenario (e.g., a rotating face). Moreover, even hardware-based techniques cannot be used. One could try to remove motion blur by directly measuring camera motion via odometers fitted in the phone. This approach can only eliminate blur due to camera motion, but not blur due to motion of objects. It is therefore necessary to look at a novel framework to address this problem.The main difficulty with removing blur from an articulated or deforming body is that blur is typically non-smooth and space-varying. Most notably, blur in these instances is characterized by occlusions. Consider for example the picture of a rotating head. The area around the nose will be the combination of a partial occlusion and disocclusion process. What makes this problem even more challenging is that only the exact model of the blur will allow the correct removal of blur without introducing visible artifacts. Another issue is that the blurry input image provides limited and low-quality data to make decisions about the 3D geometry of an object, its 3D motion trajectory, and its texture. Thus, the challenge is that the process is highly nonlinear, one needs to determine its model with high precision, and there is only limited and ambiguous information (the blurry input image) to make such decisions.With these challenges in mind, we propose to study this category of blind deconvolution problems with a model-based approach and by exploiting user interaction and efficient search in parameter space.
-