Project

Back to overview

Gradient-domain Monte Carlo Rendering

English title Gradient-domain Monte Carlo Rendering
Applicant Zwicker Matthias
Number 163045
Funding scheme Project funding (Div. I-III)
Research institution Department of Computer Science University of Maryland
Institution of higher education University of Berne - BE
Main discipline Information Technology
Start/End 01.01.2016 - 31.12.2019
Approved amount 421'804.33
Show all

Keywords (4)

Image Synthesis; Computer Science; Computer Graphics; Sampling and Reconstruction

Lay Summary (German)

Lead
In diesem Projekt entwickeln wir neue Algorithmen, um schneller mittels Computerberechnungen realistische Bilder zu erzeugen. Zu den Andwendungen solcher Algorithmen gehören zum Beispiel Filmproduktion, Computerspiele, medizinische Bildgebung, wissenschaftliche Visualisierung oder geografische Informationssysteme.
Lay summary
Computergrafik befasst sich mit der Berechnung von Bildern mittels Computer. Forschungserfolge in diesem Gebiet haben dazu beigetragen, dass heute computergenerierte Bilder allgegenwärtig sind. Zu den Andwendungen der Computergrafik gehören zum Beispiel Filmproduktion, Computerspiele, medizinische Bildgebung, wissenschaftliche Visualisierung oder geografische Informationssysteme. In diesem Projekt untersuchen wir Herausforderungen, welche die Möglichkeiten der Computergrafik immer noch zurückhalten. Speziell entwickeln wir Algorithmen zur realistischen Bildsynthese. Hier geht es darum, auf Grund von dreidimensionalen Szenen, welche im Computer gespeichert sind, Bilder zu generieren. Bilder sind eine Darstellung der Verteilung von Licht. Während man Licht in der physikalischen Welt als kontinuierliche Grösse beschreiben kann, muss man zur Darstellung mittels Computer auf diskrete Repräsentationen zurückgreifen. Dies führt zum Problem der Abtastung, wie zum Beispiel die Abtastung von Lichtwerten auf dem Pixelgitter eines digitalen Bildes. In unserer Forschung entwickeln wir neue Strategien zur Abtastung von Licht, um schneller Bilder zu berechnen, welche keine sichtbaren Artefakte mehr aufweisen. Solche Algorithmen werden neue Anwendungen ermöglichen, und existierende Anwendungen können schneller und mit wesentlich weniger Rechenaufwand gelöst werden.
Direct link to Lay Summary Last update: 25.09.2015

Lay Summary (English)

Lead
In this project we are developing new algorithms to generate realistic images using computer simulations. Applications of such algorithms include movie production, computer games, medical imaging, scientific visualization, or geographic information systems.
Lay summary
Computer graphics is concerned with using computers to create images. Research in this area has been so successful that computer generated images have become ubiquitous. Applications of computer graphics range from the entertainment industry, communication technologies, medical visualization and scientific applications to everyday tools like digital maps. In this project we tackle research challenges that are still limiting the capabilities of computer graphics technology. In particular, we will develop more efficient algorithms for image synthesis. Image synthesis is the task to create images of three dimensional scenes that are stored digitally in computers. Images represent distributions of light. While light in the physical world can be interpreted as a continuous quantity, it needs to be represented discretely for computer processing. This leads to the problem of sampling, which is at the core of this proposal. Computer graphics deals with various forms of sampled light to achieve realistic and efficient image synthesis. This includes, for example, the notion of light paths that store the amount of light transmitted along paths including several reflections at surfaces; the concept of transport operators that describe how light is passed between pairs of surface points; or radiance distributions that represent the light that is reflected in each direction at each surface point. In this project we are investigating efficient techniques to sample these quantities using correlated samples, which we relate to differences between image pixels called gradients. We will show in a theoretical analysis how correlated samples can reduce the error in image synthesis. We will develop improved algorithms for image synthesis that reduce the computation time to obtain results without visual artifacts, and that will enable more natural and effective interactive applications.
Direct link to Lay Summary Last update: 25.09.2015

Responsible applicant and co-applicants

Employees

Publications

Publication
Learning to Importance Sample in Primary Sample Space
Zheng Quan, Zwicker Matthias (2019), Learning to Importance Sample in Primary Sample Space, in Computer Graphics Forum, 38(2), 169-179.
Light transport simulation in the gradient domain
Hua Binh-Son, Gruson Adrien, Zwicker Matthias, Hachisuka Toshiya (2018), Light transport simulation in the gradient domain, in SIGGRAPH Asia 2018 Courses, Tokyo, JapanAssociation for Computing Machinery, New York, NY, USA.
Temporal gradient-domain path tracing
Manzi Marco, Kettunen Markus, Durand Frédo, Zwicker Matthias, Lehtinen Jaakko (2016), Temporal gradient-domain path tracing, in ACM Transactions on Graphics, 35(6), 1-9.
Regularizing Image Reconstruction for Gradient-Domain Rendering with Feature Patches
Manzi M., Vicini D., Zwicker M. (2016), Regularizing Image Reconstruction for Gradient-Domain Rendering with Feature Patches, in Computer Graphics Forum, 35(2), 263-273.

Collaboration

Group / person Country
Types of collaboration
MIT CSAIL United States of America (North America)
- in-depth/constructive exchanges on approaches, methods or results
Aalto University, School of Science Finland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
Eurographics 2019 Talk given at a conference Learning to Importance Sample in Primary Sample Space 06.05.2019 Genova, Italy Zwicker Matthias;
SIGGRAPH Asia 2018 Talk given at a conference Light Transport Simulation in the Gradient Domain 04.12.2018 Tokyo, Japan Zwicker Matthias;
SIGGRAPH Asia 2016 Talk given at a conference Temporal gradient-domain path tracing 05.12.2016 Macao, China Manzi Marco;
Eurographics 2016 Talk given at a conference Regularizing Image Reconstruction for Gradient‐Domain Rendering with Feature Patches 09.05.2016 Lisbon, Portugal Zwicker Matthias; Manzi Marco;


Use-inspired outputs

Associated projects

Number Title Start Funding scheme
143886 Efficient Sampling and Reconstruction for Image Synthesis 01.01.2013 Project funding (Div. I-III)

Abstract

Efficiently creating realistic images by simulating light transport in virtual scenes has long been a core research problem in computer graphics. Considerable progress over the years has enabled a wealth of applications in the areas of digital entertainment, communication, data visualization, or scientific research. Despite these successes, image synthesis using light transport simulation often requires minutes to hours of computation time, and interactive applications are limited to simple effects or highly approximate solutions. As a consequence, the academic research community continues to explore novel algorithms towards the grand vision to create photo-realistic images including complex light transport effects fast enough for interactive applications.With support from the Swiss National Science Foundation, the Computer Graphics Group at the University of Bern has made significant contributions to the advancement of image synthesis, or rendering algorithms, over the last few years. Here we propose to build on and further develop the main insights that we achieved in the last funding period, and which had an immediate impact in the research community as well as in industry. The results of this project will contribute one more step towards efficient rendering algorithms that will ultimately provide interactive, photo-realistic imagery in a broad range of applications.The goal of image synthesis using light transport simulation is to compute images of virtual, three-dimensional environments such that, if it were possible to capture photographs of equivalent physical environments, the simulated images would be visually indistinguishable from the photographs. In an actual digital camera, the brightness of a pixel is determined by measuring the photon energy incident over the area of the pixel on the sensor. Photons can be thought of as particles that scatter in the physical environment with a certain randomness, tracing out paths from light sources to the camera lens and ultimately onto the sensor, where they are absorbed. The same intuition underlies Monte Carlo methods, a broad class of techniques to simulate light transport and image formation using virtual environments and virtual cameras. They construct light paths (Monte Carlo samples) with a certain randomness and measure their contributions over virtual sensor pixels. As an overarching research objective, we are striving to develop algorithms that reduce image errors to a minimum under a given sample budget.In this project, we will develop novel approaches for gradient-domain Monte Carlo rendering, a recently proposed, innovative strategy that promises to be useful for a wide range of rendering algorithms. A key intuition underlying gradient-domain Monte Carlo rendering is that image space gradients between neighboring pixels, that is, pixel differences, can be sampled with little variance (``noise'') by sampling pairs of paths through the corresponding pixels such that they are close to each other in path space. Such paths tend to make similar contributions to the image, and hence they contribute small values to the gradient. By combining the estimated gradients with a noisy image using an L_2 Poisson reconstruction step, one obtains unbiased rendering results with significantly lower noise and lower error compared to sampling the image pixels only, as in conventional techniques. Gradient-domain rendering was initially proposed specifically for Metropolis sampling, where it significantly lowers the error of rendered images at equal computation time. However, the generalizability and applicability of gradient sampling and reconstruction to other Monte Carlo rendering algorithms, like path tracing and its variants, has not been well explored. In addition, a theoretical understanding why gradient-domain rendering can be beneficial is largely absent. Finally, the application of advanced rendering techniques such as importance sampling, or adaptive sampling and reconstruction, has not been investigated for gradient-domain rendering at all.We will address the crucial gaps in the theoretical understanding and analysis of gradient-domain rendering, and we will develop novel algorithms that will leverage the benefits of gradient sampling and reconstruction in a variety of scenarios. Specifically, we will develop a frequency analysis of gradient-domain Monte Carlo rendering that show precisely under which circumstances gradient sampling and reconstruction is beneficial compared to conventional Monte Carlo sampling. In addition, we will develop gradient-domain algorithms for path tracing, bidirectional path tracing, and other path sampling strategies. We will investigate the applicability of importance sampling and multiple importance sampling in gradient domain rendering. We will consider specific light transport effects such as motion blur and depth-of-field, and explore how their specific properties can be exploited in gradient-domain rendering. Finally, we will develop techniques for adaptive sampling and reconstruction. In summary, there is a significant research agenda of largely unexplored issues in gradient-domain rendering. We have significantly contributed to initial results, and this project will allow us to keep leading the way in this promising area.
-