Project

Back to overview

Multidimensional Sampling for Image Synthesis

English title Multidimensional Sampling for Image Synthesis
Applicant Zwicker Matthias
Number 127166
Funding scheme Project funding (Div. I-III)
Research institution Institut für Informatik Universität Bern
Institution of higher education University of Berne - BE
Main discipline Information Technology
Start/End 01.10.2009 - 30.09.2012
Approved amount 319'964.00
Show all

Keywords (6)

Computer Science; Computer Graphics; Image Synthesis; Sampling and Reconstruction; 3D computer graphics; realistic and interactive rendering

Lay Summary (English)

Lead
Lay summary
Computer graphics is concerned with using computers to create images. Research in this area has been so successful that computer generated images have become ubiquitous. Applications of computer graphics range from the entertainment industry, communication technologies, medical visualization and scientific applications to everyday tools like digital maps. In this project we tackle research challenges that are still limiting the capabilities of computer graphics technology. In particular, we will develop more efficient algorithms for image synthesis. Image synthesis is the task to create images of three dimensional scenes that are stored digitally in computers. Images represent distributions of light. While light in the physical world can be interpreted as a continuous quantity, it needs to be represented discretely for computer processing. This leads to the problem of sampling, which is at the core of this proposal. Computer graphics deals with various forms of sampled light to achieve realistic and efficient image synthesis. This includes, for example, the notion of light paths that store the amount of light transmitted along paths including several reflections at surfaces; the concept of transport operators that describe how light is passed between pairs of surface points; or radiance distributions that represent the light that is reflected in each direction at each surface point. All three concepts require multiple numbers to identify each sample. Light paths are identified by their sequence of intersection points with surfaces. A sample of a transport operator can be identified by a pair of surface points. A surface point is identified by two parameters, i.e., a surface is two-dimensional, hence a pair of points is four-dimensional. Radiance distributions are over surface locations and directions, therefore they are also four-dimensional. This is to explain why multidimensional sampling plays a central role in image synthesis.In this project we focus on two topics, realistic image synthesis and light transport for interactive rendering, which at their core involve multidimensional sampling issues. We provide an analysis of both topics from the perspective of multidimensional sampling and develop more efficient and flexible algorithms. Improvements of these core algorithms for image synthesis will find broad applications in computer graphics. They will reduce the resources required for realistic image synthesis, and enable more natural and effective interactive applications.
Direct link to Lay Summary Last update: 21.02.2013

Responsible applicant and co-applicants

Employees

Publications

Publication
Adaptive Rendering using Non-Local Means Filtering
Rousselle Fabrice, Knaus Claude, Zwicker Matthias (2012), Adaptive Rendering using Non-Local Means Filtering, in ACM Transactions on Graphics, 31(6), 195:1-195:11.
Adaptive sampling and reconstruction using greedy error minimization
Rousselle Fabrice, Knaus Claude, Zwicker Matthias (2011), Adaptive sampling and reconstruction using greedy error minimization, in ACM Trans. Graph., 30(6), 159:1-159:12.
Progressive photon beams
Jarosz Wojciech, Nowrouzezahrai Derek, Thomas Robert, Sloan Peter-Pike, Zwicker Matthias (2011), Progressive photon beams, in ACM Trans. Graph., 30(6), 181:1-181:12.
Progressive Photon Mapping: A Probabilistic Approach
Knaus C, Zwicker M (2011), Progressive Photon Mapping: A Probabilistic Approach, in ACM TRANSACTIONS ON GRAPHICS, 30(3), 25:1-25:13.

Collaboration

Group / person Country
Types of collaboration
Blue Sky Studios United States of America (North America)
- Industry/business/other use-inspired collaboration
Disney Research Zurich Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
ACM SIGGRAPH Asia 2012 Talk given at a conference Adaptive Rendering with Non-local Means Filtering 29.11.2012 Singapore, Singapore Rousselle Fabrice;
ACM SIGGRAPH Asia 2011 Talk given at a conference Adaptive sampling and reconstruction using greedy error minimization 12.12.2011 Hong Kong, China Rousselle Fabrice;
ACM SIGGRAPH 2011 Talk given at a conference Progressive Photon Mapping: A Probabilistic Approach 07.08.2011 Vancouver, Canada Zwicker Matthias;


Awards

Title Year
Eurographics PhD Award 2016

Associated projects

Number Title Start Funding scheme
143886 Efficient Sampling and Reconstruction for Image Synthesis 01.01.2013 Project funding (Div. I-III)

Abstract

Computer graphics is concerned with using computers to create images. Applications of computer graphics range from the entertainment industry, communication technologies, medical visualization and scientific applications to everyday tools like digital maps. In this proposal we identify research challenges that are still limiting the capabilities of computer graphics technology. In particular, we will develop novel algorithms for image synthesis by studying this problem from the perspective of multidimensional sampling.We focus on two topics, efficient algorithms for realistic image synthesis and light transport for interactive rendering, which at their core involve multidimensional sampling issues. We provide an analysis of both topics from the perspective of multidimensional sampling, describe specific research challenges, and lay out our goals to develop more efficient and flexible algorithms. Improvements of these core algorithms for image synthesis will find broad applications in computer graphics. They will reduce the resources required for realistic image synthesis, and enable more natural and effective interactive applications.Efficient Algorithms for Realistic Image Synthesis: The goal of realistic image synthesis is to generate computer images that are indistinguishable from images of the physical world as perceived by the human visual system or as captured by photographic cameras. This is achieved by faithfully simulating, based on physical models, how light is transported through three dimensional scenes and captured by a sensor. Light transport simulations, however, are computationally very expensive. The production of realistic film sequences, for example, requires weeks of computation time on large computer clusters. Therefore, the quest for more efficient algorithms for realistic image synthesis is an ongoing topic in computer graphics research. Such algorithms could significantly lower the cost of infrastructure and energy consumption involved today in industrial-scale image production.In this project we will develop more efficient methods for light transport simulation. Our main idea is to exploit the multidimensional properties of light paths to achieve this. The proposed research builds on our recent work where we introduced a novel approach to image synthesis based on multidimensional adaptive sampling instead of Monte Carlo path tracing. By leveraging multidimensional properties of light paths, this approach can produce higher quality images at equal computation times than previous algorithms. In the proposed project we will further develop this novel approach to image synthesis. Our initial proof-of-concept algorithm is limited to certain effects such as simulating depth of field and motion blur. It also relied on an experienced user to tune its parameters. The goal of this project is to generalize our framework to be more widely applicable, to remove the need for user intervention, and to further improve its efficiency.Light Transport for Interactive Rendering: Computer graphics applications are often interactive; i.e., they need to respond to user input with little delay and generate images accordingly at a rate of several frames per second. Yet, obtaining images that are as realistic as possible is important for many applications. Precomputed radiance transfer is an elegant solution to enable light transport simulations at interactive rates. The core idea is to precompute a transport operator that describes the transport of light between each pair of points in the scene. Given the transport operator for a scene, it is possible to generate images from any viewpoint and under arbitrary illumination in fractions of a second.The long term vision of our research is to obtain algorithms that provide all visual effects of global light transport in interactive applications. Towards this goal, we will research effective representations of the transport operator and methods to sample and store it efficiently. We have recently introduced a novel representation that is based on multilevel scattered data interpolation and meshless finite elements. The core idea of this approach is to decouple the representation of scene geometry from the representation of the transport operator. As a consequence our approach can handle more complex scenes than previously possible. In this project we will extend and improve our method in several ways. We will allow for more general surface reflection properties, dynamic geometry, and participating media. We also develop new data compression schemes, make precomputation more efficient, and improve the rendering performance.
-