social interaction modeling; organizational behavior; nonverbal behavior; computational social science; social computing; social psychology; perceptual computing; ubiquitous computing
Kenneth Funes, Laurent Nguyen, Daniel Gatica-Perez, Jean-Marc Odobez (2013), A semi-automated system for accurate gaze coding in natural dyadic interactions, in
Proceedings of the 15th ACM on International conference on multimodal interaction, Sydney, Australia.
Marcos-Ramiro Alvaro, Pizarro-Perez Daniel, Marron-Romera Marta, Nguyen Laurent, Gatica-Perez Daniel (2013), Body communicative cue extraction for conversational analysis, in
IEEE International Conference on Automatic Face and Gesture Recognition (FG), Shanghai.
Sanchez-Cortes D., Aran O., Jayagopi D., Mast M. Schmid, Gatica-Perez D. (2013), Emergent leaders through looking and speaking: from audio-visual data to multimodal recognition, in
Journal on Multimodal User Interfaces, in press, 7(1-2), 39-53.
Frauendorfer Denise, Schmid Mast Marianne (2013), Hiring gender-occupation incongruent applicants: The positive impact of recruiter interpersonal sensitivity, in
Journal of Personnel Psychology, 12, 182-188.
Do Trinh-Minh-Tri, Kalimeri Kyriaki, Lepri Bruno, Pianesi Fabio, Gatica-Perez Daniel (2013), Inferring social activities with mobile sensor networks, in
ACM International Conference on Multimodal Interaction (ICMI), Sydney.
Chittaranjan G., Blom J., Gatica-Perez D. (2013), Mining Large-Scale Smartphone Data for Personality Studies, in
Personal and Ubiquitous Computing, 17(3), 433-450.
Nguyen Laurent, Marcos-Ramiro Alvaro, Marron-Romera Marta, Gatica-Perez Daniel (2013), Multimodal analysis of body communication cues in employment interviews, in
ACM International Conference on Multimodal Interaction (ICMI), Sydney.
Aran Oya, Gatica-Perez Daniel (2013), One of a kind: Inferring personality impressions in meetings, in
ACM International Conference on Multimodal Interaction (ICMI), Sydney.
Voida Stephen, Choudhury Tanzeem, Gay Geri, Mathews Mark, Adams Phil, Rabbi Mashfiqui, Pollack JP, Chi Mengxi, Green Mathew, Lu Hong, Lane Nic, Mu Lin, Campbell Andrew (2013), Personal informatics can be stressful: Collecting, reflecting, and embedding stress data in personal informatics, in
CHI Personal Informatics Workshop , Paris.
Sanchez-Cortes D., Aran O., Schmid Mast M., Gatica-Perez D. (2012), A Nonverbal Behavior Approach to Identify Emergent Leaders in Small Groups, in
IEEE Transactions on Multimedia, 14(3-2), 816-832.
Sanchez-Cortes Dairazalia, Motlicek Petr, Gatica-Perez Daniel (2012), Assessing the impact of language style on emergent leadership perception from ubiquitous audio, in
International Conference on Mobile and Ubiquitous Multimedia (MUM), Ulm.
Jayagopi Dinesh, Sanchez-Cortes Dairazalia, Otsuka Kazuhiro, Yamato Junji, Gatica-Perez Daniel (2012), Linking speaking and looking behavior patterns with group composition, perception, and performance, in
ACM International Conference on Multimodal Interaction (ICMI), Santa Monica.
Lu H., Rabbi M., Chittaranjan G., Frauendorfer D., Mast M., Campbell A., Gatica-Perez D., Choudhury T. (2012), StressSense: Detecting Stress in Unconstrained Acoustic Environments using Smartphones, in
Proc. ACM Int. Conf. on Ubiquitous Computing, Pittsburgh.
Laurent Nguyen, Jean-Marc Odobez, Daniel Gatica-Perez (2012), Using self-context for multimodal detection of head nods in face-to-face interactions, in
Proceedings of the 14th ACM international conference on Multimodal interaction, Santa Monica, USA.
Sanchez-Cortes D., Aran O., Gatica-Perez D. (2011), An Audio Visual Corpus for Emergent Leader Analysis, in
ICMI Workshop on Multimodal Corpora for Machine Learning, 14(3-2), 14(3-2).
Aran O., Gatica-Perez D. (2011), Analysis of Group Conversations: Modeling Social Verticality, Springer, London, 293-322.
Chittaranjan Gokul, Aran Oya, Gatica-Perez Daniel (2011), Exploiting Observers' Judgments for Nonverbal Group Interaction Analysis, in
IEEE Int. Conf. on Automatic Face and Gesture Recognition, Santa Barbara.
Schmid Mast M., Frauendorfer D., Popovic L. (2011), Self-promoting and modest job applicants in different cultures, in
Journal on Personnel Psychology, 10, 70-77.
Chittaranjan G., Blom J., Gatica-Perez D. (2011), Who is Who with Big-Five: Analyzing and Classifying Personality Traits with Smartphones, in
Proc. IEEE Int. Conf. on Wearable Computers, San Francisco.
Nguyen Laurent, Frauendorfer Denise, Schmid Mast Marianne, Gatica-Perez Daniel, Hire me: Computational inference of hirability in employment interviews based on nonverbal behavior, in
IEEE Transactions on Multimedia, 00.
Frauendorfer Denise, Schmid Mast Marianne, Nguyen Laurent, Gatica-Perez Daniel, Nonverbal social sensing in action: Unobtrusive recording and extracting of nonverbal behavior in social interactions illustrated with a research example, in
Journal of Nonverbal Behavior , 00.
Frauendorfer Denise, Schmid Mast Marianne, The impact of nonverbal behavior in the job interview, in
Nonverbal Communication, 00.
Gatica-Perez Daniel, Aran Oya, Jayagopi Dinesh, Understanding Small Groups, in Magnenat-Thalmann Nadia, Burgoon Judee, Vinciarelli Alessandro, Pantic Maja (ed.), Cambridge University Press, UK, 00.
Do Trinh-Minh-Tri, Gatica-Perez Daniel, Where and what: Using smartphones to predict next locations and applications in daily life, in
Pervasive and Mobile Computing (PMC).
The understanding of organizational behavior has been a fundamental goal in social sciences in recent years. The development of theories and models to explain and improve key aspects of organizational life has an undeniable value given the increasing influence that people’s jobs’ scope and demands, their associated styles and rituals, and the resulting professional and personal relationships have on their lives. Face-to-face communication remains the primary form of social interaction in the workplace, and thus nonverbal communication plays a crucial role in organizational behavior. While spoken language constitutes a strong communication channel, a wealth of information is conveyed nonverbally in parallel to the spoken words, through tone of voice and prosody, and through proximity, body gestures and postures, gaze, and facial expressions. Nonverbal behavior (NVB) has different functions in social interactions. It not just serves to express emotions but also to signal attention, convey attitudes about friendliness or dominance, and reveal personality characteristics such as shyness or extraversion. Furthermore, the establishment and evolution of multiple organizational constructs at the level of individuals - including judgments of personality, status, and competence - and groups - the emergence of leadership, cohesion, and performance - are the often measurable outcomes of the myriad of micro-level face-to-face conversations with peers, superiors, and subordinates, where nonverbal communicative processes intervene through the (conscious or unconscious) display and interpretation of NVB.This interdisciplinary research program proposes to significantly advance the state-of-the-art in perceptual computing, ubiquitous computing, and social psychology for the computational sensing and analysis of NVB of individuals and groups in organizational scenarios at large-scale. More specifically, we aim to address three key interrelated aspects of behavior in organizations, namely leadership, personality, and performance. These behavioral constructs are not only ubiquitous at work but also key in the formation and maintenance of relationships, and ultimately in the success of individuals and teams. This project takes a novel and broad view on the subject. Through a carefully designed collaborative structure and common research objectives, we will study the relations between short-term and long-term NVB and the emergence of leadership, the assessment of personality, and the prediction of job performance, as well as the interplay among these constructs, by analyzing dyadic and small group interaction both in the laboratory and in real-life, using automatic methods and involving multiple sensors (cameras, microphones, and wearable devices). In social science we aim at advancing theories and empirical research on the interplay between first impressions, NVB, performance, and personality in job interviews and job execution; and between personality traits and exhibited NVB of emergent leaders in small groups over long-term interaction. In computer science, we aim to develop new computational methods for automatic extraction of NVB from face-to-face interaction from audio, video, and mobile sensors, and to design new computational models for recognition and discovery of social constructs from brief and long-term observations of nonverbal behavior, including leadership in small groups, personality traits in the workplace, and job performance. The proposed research program addresses a new interdisciplinary domain, which has been recognized as disruptive and capable of opening entirely new avenues for behavioral analysis research in psychology and perceptual and ubiquitous computing, and provides a timely opportunity to train young scientists on the interplay between the human and computational facets of social interaction analysis.