Project

Back to overview

Interactive Shape Deformation and Animation

Applicant Sorkine-Hornung Olga
Number 137879
Funding scheme Project funding
Research institution Professur für Informatik ETH Zürich
Institution of higher education ETH Zurich - ETHZ
Main discipline Information Technology
Start/End 01.11.2011 - 31.10.2014
Approved amount 208'112.00
Show all

All Disciplines (2)

Discipline
Information Technology
Mathematics

Keywords (4)

computer graphics; digital shape modeling; computer animation; human-computer interfaces for an

Lay Summary (English)

Lead
Lay summary
Developing methods for interactive digital shape design and animation is one of the core tasks of computer graphics and geometric modeling research. Digital geometric models are used in many areas, such as: engineering design and product prototyping, medical simulation and planning, architecture, special effects in films, virtual reality and games. Recent advances in 3D scanning technology enable to obtain 3D geometric data of real-world objects easily and cheaply, such that it becomes available not only to academics and professionals but the general public as well (e.g., commercial products like Microsoft's Kinect). Once a digital representation of a shape is obtained, it usually needs to be processed and modified to suit the needs of a particular application. Designers and artists may want to explore various shape changes and deformations to reach the final aesthetic and functionality; prosthetics prototypes need to be adjusted to the particular patient's geometry, and digital characters are brought to life by animators. However, designing and animating shapes on a computer significantly differs from manipulation of actual physical objects. Most tools for digital shape editing require professional training and time-consuming, tedious work. They typically have a complex user interface and demand visual artistic skills and substantial technical knowledge to operate. Research in geometric modeling and animation has seen significant progress in developing better algorithms and interfaces, yet current techniques remain to be either heavily dependent on expert user skills for providing proper input or are not efficient enough for real-time manipulation of complex digital shapes.

The goal of this project is to develop novel algorithms and user interfaces for shape deformation and animation that are simple and intuitive to use, automate difficult parts of the process and are fast enough for handling high-resolution digital shapes in real time. We will explore shape modeling mechanisms that use very fast and parallelizable computation. The biggest challenge here is to come up with appropriate mathematical formulation that models the spread of influence of the action that the user applies to a point on a shape (such as pulling, rotation, twisting, etc.). The second challenge is to infer the missing degrees of freedom, namely, if the user pulls on one part of the shape, what should happen to other parts of the shape, how will they respond to the impact of the user's action in an intuitive and meaningful way? Traditionally, all this is done by manually sculpting the shape on the computer, in many cases moving every sampled point on the surface. This is a time-consuming tedious process, and we wish to replace it with automatic procedures. Additionally, we will develop an affordable and versatile physical interface that enables posing and manipulating digital objects by interacting with an actual physical representation. This physical interface can be thought of as a "Lego" set of controls of the modeling system which can be combined and connected to form arbitrary  configurations, for example to mimic a human skeleton, a spider, a quadruped and so on. The user will then be able to deform and pose this physical representation, and the system will translate the actions onto the digital shape. Such interface frees the user from the necessity to mentally map the 2D projected image on the screen to 3D, provides a tangible object to interact with and should be accessible to both professionals and casual users.

The long-term vision of this project is to bring digital shape media to the public and make 3D shapes as accessible and common as digital images, videos and music are today. We aim to design the necessary representations and tools for working with 3D geometric content that would allow anybody to communicate using not just images but also dynamic 3D objects.

Direct link to Lay Summary Last update: 21.02.2013

Responsible applicant and co-applicants

Employees

Publications

Publication
A Simple Method for Correcting Facet Orientations in Polygon Meshes Based on Ray Casting
Takayama Kenshi, Jacobson Alec, Kavan Ladislav, Sorkine-Hornung Olga (2014), A Simple Method for Correcting Facet Orientations in Polygon Meshes Based on Ray Casting, in Journal of Computer Graphics Techniques (JCGT), 3(4), 53-63.
Accurate and Efficient Lighting for Skinned Models
Tarini Marco, Panozzo Daniele, Sorkine-Hornung Olga (2014), Accurate and Efficient Lighting for Skinned Models, in Computer Graphics Forum, 33(2), 421-428.
Consistently Orienting Facets in Polygon Meshes by Minimizing the Dirichlet Energy of Generalized Winding Numbers
Takayama Kenshi, Jacobson Alec, Kavan Ladislav, Sorkine-Hornung Olga (2014), Consistently Orienting Facets in Polygon Meshes by Minimizing the Dirichlet Energy of Generalized Winding Numbers.
Facial Performance Enhancement using Dynamic Shape Space Analysis
Bermano Amit, Bradley Derek, Zünd Thabo Beeler Fabio, Nowrouzezahrai Derek, Baran Ilya, Sorkine-Hornung Olga, Pfister Hanspeter, Sumner Robert, Bickel Bernd, Gross Markus (2014), Facial Performance Enhancement using Dynamic Shape Space Analysis, in ACM Transactions on Graphics, 33(2), 13.
Fast and Memory-Efficient Topological Denoising of 2D and 3D Scalar Fields
Günther David, Jacobson Alec, Reininghaus Jan, Seidel Hans-Peter, Sorkine-Hornung Olga, Weinkauf Tino (2014), Fast and Memory-Efficient Topological Denoising of 2D and 3D Scalar Fields, in IEEE Transactions on Visualization and Computer Graphics (Proceedings of IEEE SciVis 2014), 20(12), 2585-2594.
Ink-and-Ray: Bas-Relief Meshes for Adding Global Illumination Effects to Hand-Drawn Characters
Sýkora Daniel, Kavan Ladislav, Čadík Martin, Jamriška Ondrej, Jacobson Alec, Whited Brian, Simmons Maryann, Sorkine-Hornung Olga (2014), Ink-and-Ray: Bas-Relief Meshes for Adding Global Illumination Effects to Hand-Drawn Characters, in ACM Transactions on Graphics, 33(2), 16.
Tangible and Modular Input Device for Character Articulation
Jacobson Alec, Panozzo Daniele, Glauser Oliver, Pradalier Cédric, Hilliges Otmar, Sorkine-Horning Olga (2014), Tangible and Modular Input Device for Character Articulation, in ACM Transactions on Graphics (proceedings of ACM SIGGRAPH), 33(4), 82.
Animation-Aware Quadrangulation
Marcias Giorgio, Pietroni Nico, Panozzo Daniele, Puppo Enrico, Sorkine-Hornung Olga (2013), Animation-Aware Quadrangulation, in Computer Graphics Forum, 32(5), 167-175.
Consistent Volumetric Discretizations Inside Self-Intersecting Surfaces
Sacht Leonardo, Jacobson Alec, Panozzo Daniele, Schüller Christian, Sorkine-Hornung Olga (2013), Consistent Volumetric Discretizations Inside Self-Intersecting Surfaces, in Computer Graphics Forum, 32(5), 147-156.
Robust Inside-Outside Segmentation using Generalized Winding Numbers
Jacobson Alec, Kavan Ladislav, Sorkine-Hornung Olga (2013), Robust Inside-Outside Segmentation using Generalized Winding Numbers, in ACM Transactions on Graphics, 32(4), 33.
Sketch-Based Generation and Editing of Quad Meshes
Takayama Kenshi, Panozzo Daniele, Sorkine-Hornung Alexander, Sorkine-Hornung Olga (2013), Sketch-Based Generation and Editing of Quad Meshes, in ACM Transactions on Graphics, 32(4), 97.
Weighted Averages on Surfaces
Panozzo Daniele, Baran Ilya, Diamanti Olga, Sorkine-Hornung Olga (2013), Weighted Averages on Surfaces, in ACM Transactions on Graphics, 32(4), 60.
A Cotangent Laplacian for Images as Surfaces
Jacobson Alec, Sorkine Olga (2012), A Cotangent Laplacian for Images as Surfaces, (757), (757).
Fast Automatic Skinning Transformations
Jacobson Alec, Baran Ilya, Kavan Ladislav, Popovic Jovan, Sorkine Olga (2012), Fast Automatic Skinning Transformations, in ACM Transactions on Graphics (proceedings of ACM SIGGRAPH), 30(4), 77.
Smooth Shape-Aware Functions with Controlled Extrema
Jacobson Alec, Weinkauf Tino, Sorkine Olga (2012), Smooth Shape-Aware Functions with Controlled Extrema, in Computer Graphics Forum (proceedings of EUROGRAPHICS/ACM SIGGRAPH Symposium on Geometry Processing), 31(5), 1577-1586.
Stretchable and Twistable Bones for Skeletal Shape Deformation
Jacobson Alec, Sorkine Olga (2011), Stretchable and Twistable Bones for Skeletal Shape Deformation, in ACM Transactions on Graphics (proceedings of ACM SIGGRAPH ASIA), 30(6), 165.

Collaboration

Group / person Country
Types of collaboration
Autonomous Systems Lab/ETH Zurich Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Research Infrastructure
Ilya Baran/Disney Research, Zurich Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Ladislav Kavan/University of Pennsylvania United States of America (North America)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Jovan Popovic/Adobe, Inc. United States of America (North America)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
- Industry/business/other use-inspired collaboration
Cédric Pradalier/Georgia Tech Lorraine France (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Kristian Bredies/Uni Graz Austria (Europe)
- in-depth/constructive exchanges on approaches, methods or results
Otmar Hilliges/ETH Zurich Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
Meeting of the London Visual Effects Society Individual talk Tangible and Modular Input Device for Character Articulation 17.09.2014 London, Great Britain and Northern Ireland Panozzo Daniele; Jacobson Alec; Sorkine-Hornung Olga;
ACM SIGGRAPH 2014 Talk given at a conference Tangible and Modular Input Device for Character Articulation 10.08.2014 Vancouver, Canada Jacobson Alec; Panozzo Daniele; Sorkine-Hornung Olga;
EUROGRAPHICS 2014 Talk given at a conference Accurate and Efficient Lighting for Skinned Models 07.04.2014 Strasbourg, France Sorkine-Hornung Olga; Panozzo Daniele;
Vision, Modeling and Visualization 2013 Talk given at a conference Reality-inspired constraints for shape modeling and editing 11.09.2013 Lugano, Switzerland Sorkine-Hornung Olga; Panozzo Daniele;
ACM SIGGRAPH 2013 Talk given at a conference Weighted Averages on Surfaces 21.07.2013 Anaheim, United States of America Sorkine-Hornung Olga; Panozzo Daniele; Diamanti Olga;
ACM SIGGRAPH 2013 Talk given at a conference Robust Inside-Outside Segmentation using Generalized Winding Numbers 21.07.2013 Anaheim, United States of America Jacobson Alec; Sorkine-Hornung Olga;
EUROGRAPHICS Symposium on Geometry Processing 2013 Talk given at a conference Animation-Aware Quadrangulation 03.07.2013 Genoa, Italy Panozzo Daniele; Sorkine-Hornung Olga;
EUROGRAPHICS Symposium on Geometry Processing 2013 Talk given at a conference Consistent Volumetric Discretizations Inside Self-Intersecting Surfaces 03.07.2013 Genoa, Italy Panozzo Daniele; Jacobson Alec; Sorkine-Hornung Olga;
Computer Graphics International 2013 Talk given at a conference Challenges in interactive mesh modeling 11.06.2013 Hannover, Germany Sorkine-Hornung Olga;
EUROGRAPHICS 2013 Talk given at a conference Interactive shape modeling: progress and challenges 06.05.2013 Girona, Spain Sorkine-Hornung Olga;
European Conference on Media Production 2012 Talk given at a conference Really real-time 3D shape modeling and animation 05.12.2012 London, Great Britain and Northern Ireland Sorkine-Hornung Olga;
ACM SIGGRAPH 2012 Talk given at a conference Fast Automatic Skinning Transformations 05.08.2012 Los Angeles, United States of America Jacobson Alec; Sorkine-Hornung Olga;
EUROGRAPHICS Symposium on Geometry Processing 2012 Talk given at a conference Smooth Shape-Aware Functions with Controlled Extrema 16.07.2012 Tallinn, Estonia Sorkine-Hornung Olga; Jacobson Alec;
Discrete Differential Geometry Workshop Talk given at a conference Weighted Averages on Surfaces 08.07.2012 Mathematical Research Institute in Oberwolfach, Germany, Germany Panozzo Daniele; Sorkine-Hornung Olga; Diamanti Olga;
ACM SIGGRAPH ASIA 2011 Talk given at a conference Stretchable and Twistable Bones for Skeletal Shape Deformation 12.12.2011 Hong-Kong, China, Hongkong Jacobson Alec; Sorkine-Hornung Olga;


Knowledge transfer events

Active participation

Title Type of contribution Date Place Persons involved
UIST 2014 Performances, exhibitions (e.g. for education institutions) 06.10.2014 Honolulu, United States Minor Outlying Islands Jacobson Alec;
Meeting of the London Visual Effects Society Talk 17.09.2014 London, Great Britain and Northern Ireland Sorkine-Hornung Olga; Jacobson Alec; Panozzo Daniele;
ACM SIGGRAPH Emerging Technologies 2014 Performances, exhibitions (e.g. for education institutions) 11.08.2014 Vancouver, Canada Jacobson Alec; Sorkine-Hornung Olga; Panozzo Daniele;


Communication with the public

Communication Title Media Place Year
New media (web, blogs, podcasts, news feeds etc.) Animate digital creatures with this pliable modular doll Wired UK International 2014
Media relations: print media, online media Animations-Figuren im Nu zum Leben erweckt 20 Minuten Western Switzerland German-speaking Switzerland Italian-speaking Switzerland 2014
Media relations: radio, television Animationsfilme leicht gemacht SRF 1, Einstein German-speaking Switzerland 2014
Media relations: radio, television Zahlenkünstlerin mit Spieltrieb SRF 1, Einstein German-speaking Switzerland 2014
Media relations: radio, television Die perfekte Formel 3sat, nano German-speaking Switzerland International 2013
Media relations: print media, online media Die simpleste Lösung ist immer die Beste Der Landbote German-speaking Switzerland 2013
Media relations: radio, television Was die Informatik in Hollywood zu suchen hat (Folge 5 der Reihe "Grosse Traeume der Wissenschaft") Swiss Radio DRS 2 German-speaking Switzerland 2012

Awards

Title Year
Eurographics Best PhD Thesis award 2014
ETH Medal for outstanding doctoral dissertation 2013
Intel Early Career Faculty Award 2013
ETH Latsis Prize 2012
Intel Doctoral Student Honor Fellowship 2012

Associated projects

Number Title Start Funding scheme
162958 Deformation and Motion Modeling using Modular, Sensor-based Input Devices 01.10.2015 Project funding

Abstract

Developing methods for interactive shape design, deformation and animation is one of the core tasks of computer graphics and geometric modeling research. Digital geometric models are used in many areas, such as: engineering design and product prototyping, medical simulation and planning, architecture, special effects in films, virtual reality and games. Recent advances in 3D scanning technology enable to obtain 3D geometric data of real-world objects easily and cheaply, such that it becomes available not only to academics and professionals but the general public as well (e.g., commercial products like Microsoft's Kinect). Once a digital representation of a shape is obtained, most applications require its modification (editing, deformation, animation) to suit the application's goals. Designers and artists may want to explore various shape deformations to reach the final aesthetic and functionality, prosthetics prototypes need to be adjusted to the particular patient's geometry, and digital characters are brought to life by animators. However, designing and animating shapes on a computer significantly differs from manipulation of actual physical objects. Most tools for digital shape editing require professional training and time-consuming, tedious work. They typically have a complex user interface and demand visual artistic skills and substantial technical knowledge to operate. Research in geometric modeling and animation has seen significant progress in developing better algorithms and interfaces, yet current techniques remain to be either heavily dependent on user skills for providing proper input or not fast enough for realtime manipulation of complex digital shapes. The goal of this project is to develop novel algorithms and user interfaces for shape deformation and animation that are simple and intuitive to use, automate difficult parts of the process and are fast enough for handling high-resolution digital shapes in real time. On the algorithmic side, we wish to explore shape deformation mechanisms that use fast and parallelizable formulations such as linear blending of transformations. The biggest challenge here is to design appropriate in influence functions that spread the action that the user applies to a control handle (pulling, rotation, etc.) to the rest of the shape. Traditionally such influence functions have been created in a tedious and involved manual procedure by "painting" them onto the surface of the shape. We aim to design a fully automatic algorithm that produces effective blending weights that adhere to the shape's features, and complement them with high-quality, smooth and intuitive deformation methods. On the interface side, our goal is two-fold: first, we wish to unify the various types of control handle mechanisms proposed so far, such as point-, region- and cage-based manipulation, as well as internal skeletons. Most current methods are optimized for one particular type of control handle. We wish to take advantage of all types of handles in one coherent framework, since each type is useful in different modeling tasks for different parts of a shape. Secondly, we plan to develop an affordable and versatile physical interface that enables posing and manipulating digital objects by interacting with an actual physical representation. This physical interface can be thought of as a "Lego" set of controls of the modeling system which can be combined and connected to form arbitrary topological configurations. The user will then be able to deform and pose this physical representation, and the system will translate the actions onto the digital shape. Such interface frees the user from the necessity to mentally map the 2D projected image on the screen to 3D, provides a tangible object to interact with and should be accessible to both professionals and casual users.
-