social computing; social psychology; ubiquitous computing; nonverbal behavior; organizational behavior; computational social science
Costa Jean M, Jung Malte F, Czerwinski Mary, Guimbetiere Françcois, Le Trinh, Choudhury Tanzeem (2018), Regulating Feelings During Interpersonal Conflicts by Changing Voice Self-perception, in Proc. ACM SIGCHI Conf. on Human Factors in Computing Systems
Oertel Catharine, Lopez J., Yu Y, Funes k., Gustafson J., Black A., Odobez J.-M. (2017), Towards Building an Attentive Artificial Listener: On the Perception of Attentiveness in Audio-Visual Feedback Tokens, in Int. Conf. on Multimodal Interactions (ICMI)
, TokyoACM, ..
Siegfried R., Yu Y., Odobez J.-M. (2017), Towards the Use of Social Interaction Conventions as Prior for Gaze Model Adaptation, in Int. Conf. on Multimodal Interactions (ICMI)
Yu Y., Funes K., Odobez J.-M. (2017), Robust and Accurate 3D Head Pose Estimation through 3DMM and Online Head Model Reconstruction, in 12th IEEE Int. Conf. on Automatic Face and Gesture Recognition (FG)
, WashingtonIEEE, ..
Nguyen Laurent Son, Ruiz-Correa Salvador, Schmid Mast Marianne, Gatica-Perez Daniel (2017), Check Out This Place: Inferring Ambiance from Airbnb Photos, in IEEE Transactions on Multimedia
Costa Jean, Adams Alexander T, Jung Malte F, Guimbretière Françcois, Choudhury Tanzeem (2017), EmotionCheck: A Wearable Device to Regulate Anxiety through False Heart Rate Feedback, in GetMobile: Mobile Computing and Communications
, 21(2), 22-25.
Muralidhar Skanda, Gatica-Perez Daniel (2017), Examining Linguistic Content and Skill Impression Structure for Job Interview Analytics in Hospitality, in Proc. Int. Conf. on Mobile and Ubiquitous Multimedia
, ACM, Stuttgart.
Hall Judith A., Back Mitja D., Nestler Steffen, Frauendorfer Denise, Schmid Mast Marianne, Ruben Mollie A. (2017), How Do Different Ways of Measuring Individual Differences in Zero-Acquaintance Personality Judgment Accuracy Correlate With Each Other?, in Journal of Personality
Muralidhar Skanda, Schmid Mast Marianne, Gatica-Perez Daniel (2017), How May I Help You? Behavior and Impressions in Hospitality Service Encounters, in Proc. ACM Int. Conf. on Multimodal Interaction
, ACM, Glasgow.
Hall J. A., Schmid Mast M., West T. (2016), Accurate interpersonal perception, in Mast M. Schmid, Hall J. A., West T. (ed.), Cambridge University Press, Cambridge MA, 3-22.
Muralidhar Skanda, Costa Jean, Nguyen Laurent Son, Gatica-Perez Daniel (2016), Dites-Moi : Wearable Feedback on Conversational Behavior, in Proc. Int. Conf. on Mobile and Ubiquitous Multimedia
, ACM, Rovaniemi.
Costa Jean, Adams Alexander T, Jung Malte F, Guimbetiere Françcois, Choudhury Tanzeem (2016), EmotionCheck: leveraging bodily signals and false feedback to regulate our emotions, in Proc. ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing
, 758-769, 758-769.
Nguyen L.S., Gatica-Perez D. (2016), Hirability in the Wild: Analysis of Online Conversational Video Resumes, in IEEE Transactions on Multimedia
, 18(7), 1422-1437.
Schmid Mast M., Latu I (2016), Interpersonal accuracy in relation to the workplace, leadership, and hierarchy, in West T., Hall J. A., Mast M. Schmid (ed.), Cambridge University Press, Cambridge MA, 270-286.
Finnerty Ailbhe N., Muralidhar Skanda, Nguyen Laurent Son, Pianesi Fabio, Gatica-Perez Daniel (2016), Stressful First Impressions in Job Interviews, in Proc. ACM Int. Conf. on Multimodal Interaction
, 325-332, ACM, Tokyo325-332.
Muralidhar Skanda, Nguyen Laurent Son, Frauendorfer Denise, Odobez Jean-Marc, Schmid Mast Marianne, Gatica-Perez Daniel (2016), Training on the Job: Behavioral Analysis of Job Interviews in Hospitality, in Proc. ACM Int. Conf. on Multimodal Interaction
, 84-91, 84-91.
Frauendorfer D., Schmid Mast M., Sanchez-Cortes D., Gatica-Perez D. (2015), Emergent Power Hierarchies and Group Performance, in International Journal of Psychology
, 50(5), 392-396.
Chavez-Martinez G, Ruiz-Correa S., Gatica-Perez D. (2015), Happy and Agreeable? Multi-Label Classification of Impressions in Social Video, in Proceedings of the International Conference on Mobile and Ubiquitous Multimedia (MUM)
Chen Y., Yu Y., Odobez J.-M. (2015), Head Nod Detection From a Full 3D Model, in Proceedings IEEE International Conference on Computer Vision (ICCV) Workshops
Nguyen L. S., Gatica-Perez D. (2015), I Would Hire You in a Minute: Thin Slices of Nonverbal Behavior in Job Interviews, in In Proceedings ACM Int. Conference on Multimodal Interaction (ICMI)
Sanchez-Cortes D., Kumano S., Otsuka K., Gatica-Perez D. (2015), In the Mood for Vlog: Multimodal Inference in Conversational Social Video, in ACM Trans. on Interactive Intelligent Systems
, 5(2), 1-24.
Marcos-Ramiro A., Pizarro D., Marron-Romera M., Gatica-Perez D. (2015), Let Your Body Speak: Communicative Cue Extraction on Natural Interaction using RGBD Data, in IEEE Trans. on Multimedia
, 17(10), 1721-1732.
Adams A.T., Costa J., Jung M. F., Choudhury T. (2015), Mindless Computing: Designing Technologies to Subtly Influence Behavior, in Proc. ACM International Joint Conference on Pervasive and Ubiquitous Computing (Ubicomp)
Okada S., Aran O., Gatica-Perez D. (2015), Personality Trait Classification via Co-Occurrent Multiparty Multimodal Event Discovery, in Proc. ACM Int. Conf. on Multimodal Interaction (ICMI)
Murphy N., Hall J. A., Schmid Mast M., Ruben M. A., Frauendrofer D., Blanch-Hartigan D., Roter D., Nguyen L. (2015), Reliability and Validity of Nonverbal Thin Slices in Social Interactions, in Personality and Social Psychology Bulletin
, 41(2), 199-213.
Gatica-Perez D. (2015), Signal Processing in the Workplace, in IEEE Signal Processing Magazine
, 32(1), 121-125.
Ruben M. A., Hall J. A., Schmid Mast M. (2015), Smiling in a Job Interview: When Less is More, in The Journal of Social Psychology
, 155(2), 107-126.
Schmid Mast M., Gatica-Perez D., Frauendorfer D., Nguyen L. S., Choudhury T. (2015), Social Sensing for Psychology: Automated Interpersonal Behavior Assessment, in Current Directions in Psychological Science
, 24(2), 154-160.
Frauendorfer D., Schmid Mast M. (2015), The Impact of Nonverbal Behavior in the Job Interview, 220-247.
Frauendorfer D., Schmid Mast M., Sutter C. (2015), To Include or not to Include? Accuracy of Personality Judgements from Resumes with and without Photographs, in Swiss Journal of Psychology
, 74, 207-215.
Marcos-Ramiro A., Pizarro-Perez D., Marron-Romera M., Gatica-Perez D. (2014), Automatic Blinking Detection towards Stress Discovery, in Proceedings of the ACM International Conference on Multimodal Interaction (ICMI)
Marcos-Ramiro A., Pizarro-Perez D., Marron-Romera M., Gatica-Perez D. (2014), Capturing Upper Body Motion in Conversation: an Appearance Quasi-Invariant Approach, in Proceedings of the ACM International Conference on Multimodal Interaction (ICMI)
Nguyen L. S., Frauendorfer D., Schmid Mast M., Gatica-Perez D. (2014), Hire Me: Computational Inference of Hirability in Employment Interviews Based on Nonverbal Behavior, in IEEE Transactions on Multimedia
, 16(4), 1018-1031.
Hall J. A., Latu I. M., Carney D., Schmid Mast M. (2014), Nonverbal communication and the vertical dimension of social relations, in Tracy J. L., Anderson C., Cheng J. T. (ed.), 325-346.
Frauendorfer D., Schmid Mast M., Nguyen L., Gatica-Perez D. (2014), Nonverbal Social Sensing in Action: Unobtrusive Recording and Extracting of Nonverbal Behavior in Social Interactions Illustrated with a Research Example, in Journal of Nonverbal Behavior
, 38(2), 231-245.
Adams P., Rahman T., Rabbi M., Voida S., Gay G., Choudhury T. (2014), Towards Personal Stress Informatics: Comparing Minimally Invasive Techniques for Measuring Daily Stress in the Wild, in Conference on Pervasive Computing technologies for Healthcare
Schmid Mast Marianne, Hall J. A., The vertical dimension of social signaling, in Pantic M., Magnenat-Thlmann N., Burgoon J. K., Vinciarelli A. (ed.).
Gatica-Perez D., Aran O., Jayagopi D. B., Understanding Small Groups, in Pantic M., Magnenat-Thalmann N., Burgoon J., Vinciarelli A. (ed.), Cambridge University Press, Cambridge.
First impressions matter. Brief interpersonal interactions with people we do not know occur everywhere and often, and the evaluations we make of others (and that others make of us) can have profound effects. When we meet somebody for the first time, we form an impression very quickly based on their nonverbal behavior (tone of voice, speech modulation, speech duration, gaze, facial expressions, body movements and postures) and the spoken words, and this impression in turn guides our behavior. Specifically, (first) impressions in the workplace can affect outcomes like being hired or promoted, and positive impressions are critical in entire sectors of the economy, including sales, service, and hospitality, where employees are frequently evaluated, rewarded, and advanced in their careers based on the impressions they convey. Customers judge quality of service based on their impressions about the service provider -- how friendly or competent or helpful.Research in organizational psychology and nonverbal communication has revealed some of the connections existing between nonverbal behavior and impressions in the workplace for some years, including links between immediacy behavior and hiring decisions or ratings of supervisors. However, and despite the fact that impressions at work are ubiquitous, much of the existing research in this domain has largely been done in the laboratory, for single organizational situations, and based on single interactions. Furthermore, one of the fundamental goals of organizational behavior research -- how to make these findings useful for training and improvement of skills by employees -- has often been disconnected from laboratory experiments given the lack of methods and tools to systematically study favorable first impressions and other related variables in the field, with multiple job-related situations, over time, and embedded with the training process.This is what UBImpressed aims to achieve through a multidisciplinary research approach involving academics in Work and Organizational Psychology, Multimodal Signal Processing, Ubiquitous Computing, and Social Computing. The project innovatively integrates nonverbal communication research with mobile sensing, perceptual computing, machine learning, and mobile visualization applications to understand (1) what nonverbal behaviors are related to conveying favorable first impressions in different domains within the organizational context; (2) which of these impression-related behavioral cues can be robustly extracted and analyzed by automatic means over multiple physical settings; and (3) how favorable first impressions in the workplace can be trained by integrating expert knowledge with automatic sensing, analysis, and visualization technology.The project will advance the state of the art regarding nonverbal communication and organizational psychology and automated analysis of human behavior, and brings about new research directions that have the potential to impact jointly in psychology and computer science. These include: the partnership with a real organization in the hospitality industry for whose members favorable first impressions are critical for success; the ecologically valid analysis of first impressions in a population for an extended period of time and under psychological and computational lenses, both in the laboratory and the field; the investigation of online video to generate first-impression data, whose analysis could be transferred to the physical world; the design of new machine perception and machine learning methodologies to extract behavioral features, discover behavioral patterns in individuals, and make automatic predictions about first impressions; the embedding of data-driven machine analysis and human expertise in training into real applications to be used by people to improve their skills; and the generation of research resources that have the potential of being widely reused by the research community.