Project

Back to overview

Reconfiguring Comics in our Digital Era

Applicant Süsstrunk Sabine
Number 180359
Funding scheme Sinergia
Research institution Laboratoire d'images et représentation visuelle EPFL - IC - ISC - IVRL
Institution of higher education EPF Lausanne - EPFL
Main discipline Interdisciplinary
Start/End 01.03.2019 - 28.02.2023
Approved amount 2'190'591.00
Show all

All Disciplines (3)

Discipline
Interdisciplinary
Visual arts and Art history
Information Technology

Keywords (7)

Comics reconfiguration; Comics analysis; Comics segmentation; Evolution of comics; Visual computing for comics; Digital comics; 3D reconstruction from comics

Lay Summary (French)

Lead
En dépit de l’essor des technologies numériques, les auteurs de bandes dessinées continuent, pour la plupart, de privilégier des techniques de production traditionnelles, tout en diffusant leurs œuvres sur une diversité croissante de supports, tels que les smartphones et les tablettes. Malheureusement, le transfert d’un support à un autre reste difficile : Soit il force une lecture case par case qui fait disparaître l’organisation visuelle d’une page complète, soit il requiert une coûteuse intervention manuelle de la part des artistes.
Lay summary

Le but de ce projet est de faciliter le transfert d'une BD d'un support à un autre. En particulier, nous viserons à :

  1. analyser l’histoire et les enjeux du processus de reconfiguration de la BD ;
  2. élaborer des algorithmes pour aider les artistes dans le processus de reconfiguration de leurs œuvres ;
  3. étudier l'impact de l’essor des technologies numériques sur les acteurs de la BD (auteurs, éditeurs, lecteurs).
Sur le plan culturel, nous évaluerons la manière dont le numérique a influencé la BD européenne, en tenant compte du fait que la transition vers les écrans n’est qu’une étape dans un processus de reconfiguration observable depuis les origines du média. Du côté de l'informatique visuelle, tandis que l'état de l'art reste confiné à la détection de cases, de textes et de boîtes délimitant des personnages, nous fournirons des segmentations détaillées d'éléments graphiques (personnages, objets) et des notions de 3D. Les algorithmes seront exploités pour reconfigurer, en collaboration avec les auteurs, une sélection de travaux de diplômes des étudiants de l’École supérieure de bande dessinée et d’illustration de Genève (ESBDi). Ces travaux serviront également à préparer une exposition interactive dans le cadre du festival BDFIL.
Direct link to Lay Summary Last update: 18.12.2018

Responsible applicant and co-applicants

Employees

Project partner

Publications

Publication
Computer Vision – ECCV 202016th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVIII
Kim Seungryong, Süsstrunk Sabine, Salzmann Mathieu (2020), Computer Vision – ECCV 202016th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVIII, in Lecture Notes in Computer Science (LNCS, volume 12373), Springer International Publishing, Cham.
DUNIT: Detection-Based Unsupervised Image-to-Image Translation
Bhattacharjee Deblina, Kim Seungryong, Vizier Guillaume, Salzmann Mathieu (2020), DUNIT: Detection-Based Unsupervised Image-to-Image Translation, in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USAIEEE , Seattle, WA, USA.
A Bilingual Lexicon for a Functional Analysis of Basic Elements of Comic’s Language
KovalivGaëlle, StuckyOlivier (2019), A Bilingual Lexicon for a Functional Analysis of Basic Elements of Comic’s Language, in Image & Narrative, 20(3), 91-107.
Reinforced Attention for Few-Shot Learning and Beyond
HongJie, FangPengfei, LiWeihao, ZhangTong, SimonChristian, HarandiMehrtash, PeterssonLars, Reinforced Attention for Few-Shot Learning and Beyond, in Computer Vision and Pattern Recognition (CVPR 2021), IEEE, Seattle, WA.
Uncertainty-aware Joint Salient Object and Camouflaged Object Detection
Li Aixuan, Zhang Jing, Lv Yunqiu, Liu Bowen, Zhang Tong, Dai Yuchao, Uncertainty-aware Joint Salient Object and Camouflaged Object Detection, in IEEE Conference on Computer Vision and Pattern Recognition 2021 (CVPR 2021), virtualIEEE, Seattle, WA, USA.

Collaboration

Group / person Country
Types of collaboration
Ecole supérieure de bande dessinée et d'illustration (ESBDi) Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
Centre BD, Lausanne Switzerland (Europe)
- Research Infrastructure

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
European Conference on Computer Vision (ECCV) 2020 Poster Volumetric Transformer Networks 23.08.2020 virtuel, Great Britain and Northern Ireland Süsstrunk Sabine; KIM Seungryong; Salzmann Mathieu;
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020 Poster DUNIT: Detection-based Unsupervised Image-to-Image Translation 16.06.2020 Virtual, United States of America Bhattacharjee Deblina; KIM Seungryong; Salzmann Mathieu;
DH-Center UNIL EPFL Inauguration day Talk given at a conference Reconfiguring Comics in our Digital Era 31.10.2019 EPFL, Lausanne, Switzerland Baroni Raphaël;
La table des matières du Moyen-Âge au XXIe siècle: sa fabrique, ses lectures Talk given at a conference La table des maitères à l'heure de la bande dessinée numérique 07.06.2019 Paris, France Kovaliv Gaëlle;
The Annual conference of the International Society for the Study of Narrative 2019 Talk given at a conference Theoretical and Methodological Issues of Narrative Reconfigurations in Comics 30.05.2019 Pampelune, Spain Baroni Raphaël; Glassey Olivier;


Knowledge transfer events

Active participation

Title Type of contribution Date Place Persons involved
Inauguration of the UNIL-EPFL dhCenter Talk 31.10.2019 Lausanne, Switzerland Baroni Raphaël;
BDFIL Performances, exhibitions (e.g. for education institutions) 12.09.2019 Lausanne, Switzerland Baroni Raphaël; Kovaliv Gaëlle;
Digital Comics. Overview of the main publication logics Talk 01.04.2019 University of Lausanne, Switzerland Stucky Olivier; Kovaliv Gaëlle;


Communication with the public

Communication Title Media Place Year
Media relations: print media, online media Bande dessinée d'un autre genre BDFIL Western Switzerland International 2019
Talks/events/exhibitions Open Days EPFL 2019 Western Switzerland International 2019

Use-inspired outputs

Abstract

By combining graphics and text, comic books are easily accessible to all ages and social classes. They are recognized as a media in their own right and constitute an important part of our cultural heritage. Over the years, comics have constantly evolved to different formats, such as from short strips in periodicals to standalone albums and to graphic novels, which provide more artistic freedom. Unfortunately, the transfer of a comic from one format to another is a time-consuming and costly process, requiring the artist to deconstruct and reconstruct his/her work for each individual publication channel. This problem has become even more acute in our digital era, where the authors, who still predominantly favor traditional production techniques, such as paper albums, must transfer their work to an increasing diversity of formats, such as smartphones and tablets, which have become a popular publication channel.The goal of this project is to facilitate the transfer of comics from one publication channel to another by assisting authors in the task of reconfiguring their narratives. To this end, our approach will combine research in the cultural studies and in visual computing. In particular, we aim to- analyze the reconfiguration process of comics for the different analog and digital publication channels while considering the expectations of the involved actors, such as artists and readers; - develop algorithms to assist artists and editors in the creative process of reconfiguring existing comics panels to a wide variety of digital and analog publication channels;- study the impact of our reconfiguration mechanisms on the actors in the comics industry.The first of these three research axes will combine our expertise in psychophysics and in cultural and social studies to define a taxonomy of the perceptually relevant elements in comics and understand the constraints inherent in reconfiguration. In the second one, we will tackle challenging visual computing tasks to segment the graphical elements in comics panels and to reason about their 3D relationships. This will help the artists in the creative process of rearranging them so as to satisfy the constraints imposed by both the diverse publication formats and the human actors. The third axis will involve additional studies and crowd experiments, including exhibits, to analyze the quality of our reconfigured comics.Our proposed research goes far beyond the state-of-the-art in comics analysis. On the cultural studies side, there is an urgent need to establish a comprehensive account of the evolution of comics in European culture, in particular with respect to the digital media. To address this need, we will not follow a purely scholarly approach, but complement our scholarly research with field surveys including interviews of some of the actors involved in this evolution. On the visual computing side, while the state-of-the-art in comics processing remains limited to detecting panels, text and bounding boxes for specific characters, we will provide detailed segmentations of generic graphical elements and notions of 3D from comics panels. To achieve our goals, we will focus on a data-driven approach that leverages a large corpus of digitized comics.Our research will involve continuous interactions between the participants from the cultural studies side and from the visual computing one. They will arise from our psychophysics experiments (EPFL) and field study (UNIL), which will aim to understand both the reconfiguration constraints imposed by human actors and the impact of our new technology. Furthermore, we will leverage the results of these studies to develop our segmentation, 3D reconstruction and retargeting algorithms. While these interactions will create dependencies between the different parts of the project, we will provide workarounds to ensure no task is delayed by the lack of progress in another.This project constitutes the first attempt at interdisciplinary research in cultural studies and visual computing for comics reconfiguration. We believe that our results will have a significant impact not only on the comics industry by helping authors diffuse their work in diverse analog and digital publication channels, but also on museography by yielding new ways to display and interact with comics in exhibitions, and on digital humanities, by offering new means of indexing large visual digital archives.
-