Comics reconfiguration; Comics analysis; Comics segmentation; Evolution of comics; Visual computing for comics; Digital comics; 3D reconstruction from comics
Kim Seungryong, Süsstrunk Sabine, Salzmann Mathieu (2020), Computer Vision – ECCV 202016th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVIII, in
Lecture Notes in Computer Science (LNCS, volume 12373), Springer International Publishing, Cham.
Bhattacharjee Deblina, Kim Seungryong, Vizier Guillaume, Salzmann Mathieu (2020), DUNIT: Detection-Based Unsupervised Image-to-Image Translation, in
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USAIEEE , Seattle, WA, USA.
KovalivGaëlle, StuckyOlivier (2019), A Bilingual Lexicon for a Functional Analysis of Basic Elements of Comic’s Language, in
Image & Narrative, 20(3), 91-107.
HongJie, FangPengfei, LiWeihao, ZhangTong, SimonChristian, HarandiMehrtash, PeterssonLars, Reinforced Attention for Few-Shot Learning and Beyond, in
Computer Vision and Pattern Recognition (CVPR 2021), IEEE, Seattle, WA.
Li Aixuan, Zhang Jing, Lv Yunqiu, Liu Bowen, Zhang Tong, Dai Yuchao, Uncertainty-aware Joint Salient Object and Camouflaged Object Detection, in
IEEE Conference on Computer Vision and Pattern Recognition 2021 (CVPR 2021), virtualIEEE, Seattle, WA, USA.
By combining graphics and text, comic books are easily accessible to all ages and social classes. They are recognized as a media in their own right and constitute an important part of our cultural heritage. Over the years, comics have constantly evolved to different formats, such as from short strips in periodicals to standalone albums and to graphic novels, which provide more artistic freedom. Unfortunately, the transfer of a comic from one format to another is a time-consuming and costly process, requiring the artist to deconstruct and reconstruct his/her work for each individual publication channel. This problem has become even more acute in our digital era, where the authors, who still predominantly favor traditional production techniques, such as paper albums, must transfer their work to an increasing diversity of formats, such as smartphones and tablets, which have become a popular publication channel.The goal of this project is to facilitate the transfer of comics from one publication channel to another by assisting authors in the task of reconfiguring their narratives. To this end, our approach will combine research in the cultural studies and in visual computing. In particular, we aim to- analyze the reconfiguration process of comics for the different analog and digital publication channels while considering the expectations of the involved actors, such as artists and readers; - develop algorithms to assist artists and editors in the creative process of reconfiguring existing comics panels to a wide variety of digital and analog publication channels;- study the impact of our reconfiguration mechanisms on the actors in the comics industry.The first of these three research axes will combine our expertise in psychophysics and in cultural and social studies to define a taxonomy of the perceptually relevant elements in comics and understand the constraints inherent in reconfiguration. In the second one, we will tackle challenging visual computing tasks to segment the graphical elements in comics panels and to reason about their 3D relationships. This will help the artists in the creative process of rearranging them so as to satisfy the constraints imposed by both the diverse publication formats and the human actors. The third axis will involve additional studies and crowd experiments, including exhibits, to analyze the quality of our reconfigured comics.Our proposed research goes far beyond the state-of-the-art in comics analysis. On the cultural studies side, there is an urgent need to establish a comprehensive account of the evolution of comics in European culture, in particular with respect to the digital media. To address this need, we will not follow a purely scholarly approach, but complement our scholarly research with field surveys including interviews of some of the actors involved in this evolution. On the visual computing side, while the state-of-the-art in comics processing remains limited to detecting panels, text and bounding boxes for specific characters, we will provide detailed segmentations of generic graphical elements and notions of 3D from comics panels. To achieve our goals, we will focus on a data-driven approach that leverages a large corpus of digitized comics.Our research will involve continuous interactions between the participants from the cultural studies side and from the visual computing one. They will arise from our psychophysics experiments (EPFL) and field study (UNIL), which will aim to understand both the reconfiguration constraints imposed by human actors and the impact of our new technology. Furthermore, we will leverage the results of these studies to develop our segmentation, 3D reconstruction and retargeting algorithms. While these interactions will create dependencies between the different parts of the project, we will provide workarounds to ensure no task is delayed by the lack of progress in another.This project constitutes the first attempt at interdisciplinary research in cultural studies and visual computing for comics reconfiguration. We believe that our results will have a significant impact not only on the comics industry by helping authors diffuse their work in diverse analog and digital publication channels, but also on museography by yielding new ways to display and interact with comics in exhibitions, and on digital humanities, by offering new means of indexing large visual digital archives.