Project

Back to overview

LIGHT FIELD MOTION AND TURBULENCE DEBLURRING

English title LIGHT FIELD MOTION AND TURBULENCE DEBLURRING
Applicant Favaro Paolo
Number 153324
Funding scheme Project funding (Div. I-III)
Research institution Institut für Informatik Universität Bern
Institution of higher education University of Berne - BE
Main discipline Information Technology
Start/End 01.01.2015 - 31.08.2018
Approved amount 183'744.00
Show all

All Disciplines (2)

Discipline
Information Technology
Mathematics

Keywords (5)

light field; turbulence blur; blind deconvolution; motion blur; image restoration

Lay Summary (Italian)

Lead
Recentemente esistono in commercio apparecchi fotografici basati su "light field imaging" che permettono usi completamente nuovi rispetto agli apparecchi tradizionali. Per esempio è possibile cambiare la messa a fuoco dopo che la foto è stata scattata, oppure ottenere la ricostruzione di oggetti in 3D da una sola foto. Queste potenzialità non sono disponibili in apparecchi tradizionali. Lo sviluppo di questa tecnologia è però ancora agli albori. Al momento non esistono algoritmi che permettano l'eliminazione di blur dovuto al moto degli oggetti nella scena o al movimento della macchina fotografica. In questo progetto proponiamo di studiare modelli e tecniche che permettano l'eliminazione automatica di blur da foto catturate con questi apparecchi.
Lay summary

Noi proponiamo di sviluppare modelli matematici che descrivano il processo di formazione del motion blur in vari apparecchi fotografici capaci di catturare light fields (sia commerciali che prototipi sviluppati in laboratorio).  Questi modelli permetteranno di sviluppare algoritmi per rimuovere l'effetto del blur e così ottenere immagini a fuoco. La metodologia seguita e' di spezzare la complessità del modello facendo varie assunzioni a mano a mano meno stringenti. In questo modo sara' anche possibile costruire algoritmi via via più complessi usando i precedenti come punto di partenza. Ci proponiamo anche di catturare un buon dataset di immagini dove si conosce sia il blur che l'immagine a fuoco e che possa essere usato per confrontare il funzionamento di algoritmi diversi.

Il nostro lavoro permetterà di usare questi apparecchi fotografici al meglio delle loro possibilità e, in particolare, in contesti dove i soggetti fotografati possono muoversi liberamente. Questo avrà applicazioni non soltanto nell'elettronica di consumo, ma anche nell'ambito scientifico, per esempio in biologia per la riproduzione di immagini di colture di tessuti in vivo al microscopio.

Direct link to Lay Summary Last update: 28.09.2014

Responsible applicant and co-applicants

Employees

Name Institute

Publications

Publication
Learning Face Deblurring Fast and Wide
JinMeiguang, HirschMichael, FavaroPaolo (2018), Learning Face Deblurring Fast and Wide, in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, IEEE Conference on Computer Vision and Pattern Recognition , Salt Lake City, USA.
Learning to Extract a Video Sequence From a Single Motion-Blurred Image
Jin Meiguang, Meishvili Givi, Favaro Paolo (2018), Learning to Extract a Video Sequence From a Single Motion-Blurred Image, in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA.
Learning to see through reflections
Jin Meiguang, Süsstrunk Sabine, Favaro Paolo (2018), Learning to see through reflections, in 2018 {IEEE} International Conference on Computational Photography, {ICCP} 2018, Pittsburgh, PA, USA,, 1-12, International Conference on Computational Photography, Pittsburgh, PA, USA1-12.
Normalized Blind Deconvolution
Jin Meiguang, Roth Stefan, Favaro Paolo (2018), Normalized Blind Deconvolution, in Computer Vision - {ECCV} 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Pro, 694-711, ECCV, Munich, Germany694-711.
Plenoptic Image Motion Deblurring
Chandramouli Paramanand, Jin Meiguang, Perrone Daniele, Favaro Paolo (2018), Plenoptic Image Motion Deblurring, in {IEEE} Trans. Image Processing, 27(4), 1723-1734.
Deep Mean-Shift Priors for Image Restoration
Bigdeli Siavash Arjomand, Jin Meiguang, Favaro Paolo, Zwicker Mattias (2017), Deep Mean-Shift Priors for Image Restoration, in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Proces, 763-772, Neural Information Processing Systems, Long Beach, CA, USA763-772.
Noise-Blind Image Deblurring
Jin Meiguang, Roth Stefan, Favaro Paolo (2017), Noise-Blind Image Deblurring, in 2017 {IEEE} Conference on Computer Vision and Pattern Recognition, {CVPR} 2017, Honolulu, HI, USA, J, 3834-3842, Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA3834-3842.
Bilayer Blind Deconvolution with the Light Field Camera
Jin Meiguang, Chandramouli Paramanand, Favaro Paolo (2015), Bilayer Blind Deconvolution with the Light Field Camera, in 2015 {IEEE} International Conference on Computer Vision Workshop, {ICCV} Workshops 2015, Santiago, C, International Conference on Computer Vision, Santiago, Chile.

Collaboration

Group / person Country
Types of collaboration
Michael Hirsch - Amazon, Tuebingen Germany (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Stefan Roth - CS Dept, TU Darmstadt Germany (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Sabine Süsstrunk, School of Computer and Comm. Sciences, EPFL Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Matthias Zwicker, Dept of Computer Science, University of Maryland United States of America (North America)
- in-depth/constructive exchanges on approaches, methods or results
- Publication

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
European Conference on Computer Vision Poster Normalized Blind Deconvolution 10.09.2018 Munich, Germany Favaro Paolo; Jin Meiguang;
Proc. of New Trends in Image Restoration and Enhancement workshop at CVPR Poster Learning Face Deblurring Fast and Wide 20.06.2018 Salt Lake City, United States of America Favaro Paolo; Jin Meiguang;
Conference on Computer Vision and Pattern Recognition Talk given at a conference Learning to Extract a Video Sequence from a Single Motion-Blurred Image 20.06.2018 Salt Lake City, United States of America Jin Meiguang; Favaro Paolo;
International Conference on Computational Photography Talk given at a conference Learning to See through Reflections 05.05.2018 Pittsburgh, United States of America Favaro Paolo;
Conference on Computer Vision and Pattern Recognition Poster Noise-blind image deblurring 23.07.2017 Honolulu, United States of America Jin Meiguang;
IEEE International Conference on Computer Vision Workshop (ICCV Inverse Rendering Workshop) Poster Bilayer Blind Deconvolution with the Light Field Camera 12.12.2015 Santiago, Chile Jin Meiguang; Favaro Paolo;


Associated projects

Number Title Start Funding scheme
165845 BLIND 3D FACE DEBLURRING 01.01.2017 Project funding (Div. I-III)

Abstract

In the past decade there has been a proliferation of efforts on the joint design of optics, imaging sensors and algorithms to go beyond the capabilities of legacy cameras. Successes to date include, for example, extending the depth of field, obtaining 3D information from a single image, changing the focus setting digitally, and reconstructing a volume from a single image in microscopy. While a wide variety of imaging architectures has been proposed in the literature, the ones that are commercially available are essentially two: the light field cameras of Raytrix (since 2010) and Lytro (since 2011), and the camera arrays of Pointgrey (the ProFUSION 25 since 2009) and Pelican Imaging (for mobile phones, expected in 2014), just to name a few. Much of the current work in light field processing focuses on obtaining depth information and on reconstructing high-resolution sharp images, as well as on increasing their dynamic range and spectrum resolution. However, very little has been done to deal with artifacts such as motion blur. Motion blur is commonly observed in legacy cameras, and even more in cellular phone cameras due to their low-light sensitivity and long exposure intervals. Unluckily, this type of degradation can dramatically lower the image quality to the point where critical information (e.g., a face or a license plate) becomes unrecognizable.Unfortunately, deblurring algorithms designed for conventional cameras cannot be applied to light field cameras. In this project we aim at covering this gap.
-