Back to overview

Next Generation Motion Capture Platform

English title Next Generation Motion Capture Platform
Applicant Hilliges Otmar
Number 189658
Funding scheme R'EQUIP
Research institution Professur für Informatik ETH Zürich
Institution of higher education ETH Zurich - ETHZ
Main discipline Information Technology
Start/End 01.09.2020 - 31.08.2021
Approved amount 450'000.00
Show all

All Disciplines (2)

Information Technology
Sport Medicine

Keywords (8)

bio mechanics; activity recognition; computer vision; motion modelling; computer graphics; dense surface reconstruction; robotics; motion capture

Lay Summary (German)

Die Bewegungserfassung steht im Mittelpunkt vieler grundlegender Probleme der künstlichen Intelligenz, der visuellen und interaktiven Datenverarbeitung sowie der Computer Grafik. Bislang erforderte die Erfassung der Bewegung menschlicher Subjekte optische Marker, die am Körper getragen wurden, und beschränkt sich meist darauf, eine Skelettdarstellung der Bewegung zu erreichen. Dieses Projekt zielt auf die Etablierung einer neuen Motion-Capture-Plattform ab, die die genaue Geometrie der Oberfläche (einschließlich der Kleidung) menschlicher Subjekte in Bewegung erfassen kann, ohne dass die Subjekte in irgendeiner Weise instrumentiert werden müssen. Dieses Erfassungssystem wird dann als Datenquelle für viele interdisziplinäre Projekte an der ETH Zürich und darüber hinaus dienen.
Lay summary

Übergeordnetes Ziel des Projekts ist es, an der ETH Zürich eine 3D scanning Platform zu etablieren, um Zugang zu bisher schwer zugänglichen detaillierten Scans von Menschen in Bewegung zu erhalten. 

Wissenschaftlicher und gesellschaftlicher Kontext:

Das Projekt wird zur Etablierung einer neuartigen Datenerfassungsplattform führen. Diese Plattform wird zunächst für die Forschung in den Bereichen KI, Computer Vision, Computergrafik, Biomechanik und darüber hinaus von den beteiligten Forschungsgruppen an der ETH Zürich genutzt werden. Die erfassten Daten werden der breiteren Forschungsgemeinschaft in Form von Datensätzen zugänglich gemacht und fördern so Innovationen in den Bereichen KI, Robotik, Augmented und Virtual Reality, Medizin und anderen Lebenswissenschaften (z.B. Motion Assessment).

Direct link to Lay Summary Last update: 24.08.2020

Responsible applicant and co-applicants

Associated projects

Number Title Start Funding scheme
182241 Stretching the Limits of In Vivo Measurement: A wireless implantable sensor technology to quantify soft tissue strain during dynamic movements 01.11.2018 Project funding
177170 4th international conference on 'Recent Advances and Controversies in the Measurement of Energy Metabolism' (RACMEM-2017) 01.10.2017 Scientific Exchanges


BACKGROUND: Optical motion capture lies at the heart of a variety of fields, including computer vision and computer graphics, human computer interaction, robotics, and the life-sciences. The current state of the art in terms of optical motion capture systems are marker-based multi-camera systems such as VICON, or OptiTrack. These systems require retro-reflective markers attached onto subjects and are limited to skeletal kinematics. However, many important research directions now require access to not only skeletal information but also highly accurate and fine-grained information about the surface (skin, clothing) as well as its deformation over time. For example, in order to guarantee safety in shared workspaces, robots require precise surface data to estimate the spatial extent of our bodies and how clothing and other non-rigid objects may interfere with their planned motion. Similarly, autonomous vehicles need to accurately predict pedestrian movement to avoid potentially harmful situations - here, highly accurate surface information, particularly relating to faces and hands, is necessary to better gauge the pedestrian’s intent in such scenarios. More generally speaking, finding highly accurate and detailed representations of human motion allows computers to perform action recognition, motion prediction, de-noising, or automatic generation of new motion at a level of fidelity that has so far remained unobtainable. In other fields, such as rehabilitation and movement biomechanics, the more detailed quantification of human motion (as compared to maker-based systems) opens perspectives for acquiring a better understanding of how certain pathologies, such as cerebral palsy or Parkinson’s disease, affect movement and resulting risks (e.g. falls, injuries) - and whether therapeutic interventions can help. Information on the body surface may further lead to new insights into aspects of the body’s musculoskeletal dynamics that have so far remained inaccessible with marker-based systems, such as the relationship between spinal skeletal motion, the shoulder blades and the involved muscles. Finally, acquisition of detailed surface information allows generation of realistic human motion, which is of great interest to the broadcasting and entertainment industries, as it allows to further streamline and improve the very expensive process of motion capturing, re-targeting, and re-rendering.PURPOSE: The purpose of this proposal is to assist the consortium’s efforts to replace and significantly upgrade several existing marker-based motion capture systems by a high-accuracy, real-time marker-less dense surface capture system. The resulting research platform will be shared across several groups at ETH and will enable innovative and novel research in the fields of analyzing and understanding human activities from imagery (computer vision, HCI), modelling deformations of musculoskeletal dynamics during tasks of daily living and sports (biomechanics, computer graphics), and understanding and modelling the interaction between dexterous robots and complex physical objects. Beyond the five research groups directly involved in the proposal, several other professorships at ETH will benefit from the system - including two new Professorships in computer vision that are currently in an advanced negotiation stage with the president of ETH (one in electrical engineering, one in computer science). Since the planned system allows for recording of novel and unique datasets, which we plan to make accessible to research groups beyond ETH, acquisition of this system will also benefit the wider research communities at large. Finally, Zurich hosts a rich eco-system of small and large companies in a variety of areas including sports and rehabilitation and medical technologies, computer vision and graphics, and in the IT and entertainment industries. The planned motion capture platform will foster collaborations between ETH and industry in these sectors, strengthen the transfer of ideas and technologies, and thus benefit science and the general public.SIGNIFICANCE: A successful upgrade to real-time high-accuracy dense surface capture system will provide a world-wide unique platform and will be central to driving forward research in the involved Professorships and beyond, advancing the state-of-the-art in computer vision, computer graphics, robotics and sports and movement bio-mechanics. Several planned and on-going research projects within the involved Professorships crucially depend on the establishment of a non-invasive dense surface motion capture platform. The upgraded dense surface motion capture platform will provide access to research questions that cannot be addressed via marker-based tracking only. An important aspect is that the new technology does not rely on surface mounted markers and as such allows not only more detailed capturing of human shape and movement but also allows to capture detailed information about interactions between humans and their environment in far greater detail than previous systems could(e.g., exact surface to surface contact, shape of objects). Moreover, the marker-free nature of the approach allows for faster and less intrusive setup times and therefore allows for recording of more subjects in less time and even allows for recording of non-humanoid motion (e.g., animals, complex soft robots) which would be difficult or impossible with traditional motion capture technologies. Finally, the shared motion-capture platform will add considerable weight to the already very strong areas of HCI, visual computing, robotics and bio-mechanics at ETH Zurich and will establish and strengthen a growing network of international experts in a variety of areas and spanning several departments.