Project

Back to overview

UBImpressed: Ubiquitous First Impressions and Ubiquitous Awareness

English title UBImpressed: Ubiquitous First Impressions and Ubiquitous Awareness
Applicant Gatica-Perez Daniel
Number 147611
Funding scheme Sinergia
Research institution IDIAP Centre du Parc Av. des Prés-Beudin 20
Institution of higher education Idiap Research Institute - IDIAP
Main discipline Information Technology
Start/End 01.01.2014 - 31.12.2017
Approved amount 1'102'800.00
Show all

All Disciplines (2)

Discipline
Information Technology
Psychology

Keywords (6)

social computing; social psychology; ubiquitous computing; nonverbal behavior; organizational behavior; computational social science

Lay Summary (French)

Lead
Brief interpersonal interactions with people we do not know occur everywhere, and the evaluations we make have profound effects. When we meet somebody, we form an impression based on their nonverbal behavior and spoken words. Impressions in the workplace affect outcomes like being hired or promoted, and are critical in entire sectors of the economy like sales, service, and hospitality, where employees are evaluated, rewarded, and promoted based on the impressions they convey.
Lay summary
La recherche dans le domaine de la psychologie des organisations a établi depuis des années les liens qui existent entre les comportements non-verbaux et les premières impressions laissées par des personnes, comme les liens entres les comportements immédiats et les décisions d'engagement et de notation par des supérieurs hiérarchiques. Néanmoins, bien que les impressions au travail soient omniprésentes, la plupart de ces recherches ont été largement effectuées en laboratoire, pour une seule situation organisationnelle et basées sur des interactions uniques. De plus, un des objectifs principaux de la recherche de ce domaine -comment rendre ces découvertes utiles pour l'entrainement et l’amélioration des compétences des employés- a souvent été déconnecté des expériences de laboratoire étant donné le manque de méthodes et d'outils pour étudier systématiquement les premières impressions favorables et autres variables qui s'y rattachent, sur le terrain et au cours du temps, dans des situations multiples et tout en étant imbriquées avec les processus d'apprentissage.

Ces éléments constituent les objectifs que le projet UBImpressed veut atteindre grâce à une
approche multi-disciplinaire impliquant des partenaires en psychologie et organisation du travail, traitement des signaux, et modélisation par ordinateur des phénomènes sociaux. Le projet intègre la recherche en communication non-verbale et la perception artificielle et au travers de capteurs mobiles, l'apprentissage automatique, et la visualisation pour comprendre (1) quelles sont les comportements non-verbaux qui transmettent des impressions favorables dans différents contextes organisationnels; (2) parmi ceux-ci, quels sont ceux qui peuvent être mesurés de manière robuste et analysés de manière automatique dans différents espaces physiques; (3) comment les impressions favorables peuvent elles être enseignées en intégrant le savoir d'experts et les technologies d'analyse automatique ainsi que de visualisation.
Direct link to Lay Summary Last update: 13.12.2013

Responsible applicant and co-applicants

Employees

Publications

Publication
Regulating Feelings During Interpersonal Conflicts by Changing Voice Self-perception
Costa Jean M, Jung Malte F, Czerwinski Mary, Guimbetiere Françcois, Le Trinh, Choudhury Tanzeem (2018), Regulating Feelings During Interpersonal Conflicts by Changing Voice Self-perception, in Proc. ACM SIGCHI Conf. on Human Factors in Computing Systems.
Towards Building an Attentive Artificial Listener: On the Perception of Attentiveness in Audio-Visual Feedback Tokens
Oertel Catharine, Lopez J., Yu Y, Funes k., Gustafson J., Black A., Odobez J.-M. (2017), Towards Building an Attentive Artificial Listener: On the Perception of Attentiveness in Audio-Visual Feedback Tokens, in Int. Conf. on Multimodal Interactions (ICMI), TokyoACM, ..
Towards the Use of Social Interaction Conventions as Prior for Gaze Model Adaptation
Siegfried R., Yu Y., Odobez J.-M. (2017), Towards the Use of Social Interaction Conventions as Prior for Gaze Model Adaptation, in Int. Conf. on Multimodal Interactions (ICMI), Glasgow.
Robust and Accurate 3D Head Pose Estimation through 3DMM and Online Head Model Reconstruction
Yu Y., Funes K., Odobez J.-M. (2017), Robust and Accurate 3D Head Pose Estimation through 3DMM and Online Head Model Reconstruction, in 12th IEEE Int. Conf. on Automatic Face and Gesture Recognition (FG), WashingtonIEEE, ..
Check Out This Place: Inferring Ambiance from Airbnb Photos
Nguyen Laurent Son, Ruiz-Correa Salvador, Schmid Mast Marianne, Gatica-Perez Daniel (2017), Check Out This Place: Inferring Ambiance from Airbnb Photos, in IEEE Transactions on Multimedia, 1.
EmotionCheck: A Wearable Device to Regulate Anxiety through False Heart Rate Feedback
Costa Jean, Adams Alexander T, Jung Malte F, Guimbretière Françcois, Choudhury Tanzeem (2017), EmotionCheck: A Wearable Device to Regulate Anxiety through False Heart Rate Feedback, in GetMobile: Mobile Computing and Communications, 21(2), 22-25.
Examining Linguistic Content and Skill Impression Structure for Job Interview Analytics in Hospitality
Muralidhar Skanda, Gatica-Perez Daniel (2017), Examining Linguistic Content and Skill Impression Structure for Job Interview Analytics in Hospitality, in Proc. Int. Conf. on Mobile and Ubiquitous Multimedia, ACM, Stuttgart.
How Do Different Ways of Measuring Individual Differences in Zero-Acquaintance Personality Judgment Accuracy Correlate With Each Other?
Hall Judith A., Back Mitja D., Nestler Steffen, Frauendorfer Denise, Schmid Mast Marianne, Ruben Mollie A. (2017), How Do Different Ways of Measuring Individual Differences in Zero-Acquaintance Personality Judgment Accuracy Correlate With Each Other?, in Journal of Personality, 1.
How May I Help You? Behavior and Impressions in Hospitality Service Encounters
Muralidhar Skanda, Schmid Mast Marianne, Gatica-Perez Daniel (2017), How May I Help You? Behavior and Impressions in Hospitality Service Encounters, in Proc. ACM Int. Conf. on Multimodal Interaction, ACM, Glasgow.
Accurate interpersonal perception
Hall J. A., Schmid Mast M., West T. (2016), Accurate interpersonal perception, in Mast M. Schmid, Hall J. A., West T. (ed.), Cambridge University Press, Cambridge MA, 3-22.
Dites-Moi : Wearable Feedback on Conversational Behavior
Muralidhar Skanda, Costa Jean, Nguyen Laurent Son, Gatica-Perez Daniel (2016), Dites-Moi : Wearable Feedback on Conversational Behavior, in Proc. Int. Conf. on Mobile and Ubiquitous Multimedia, ACM, Rovaniemi.
EmotionCheck: leveraging bodily signals and false feedback to regulate our emotions
Costa Jean, Adams Alexander T, Jung Malte F, Guimbetiere Françcois, Choudhury Tanzeem (2016), EmotionCheck: leveraging bodily signals and false feedback to regulate our emotions, in Proc. ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing, 758-769, 758-769.
Hirability in the Wild: Analysis of Online Conversational Video Resumes
Nguyen L.S., Gatica-Perez D. (2016), Hirability in the Wild: Analysis of Online Conversational Video Resumes, in IEEE Transactions on Multimedia, 18(7), 1422-1437.
Interpersonal accuracy in relation to the workplace, leadership, and hierarchy
Schmid Mast M., Latu I (2016), Interpersonal accuracy in relation to the workplace, leadership, and hierarchy, in West T., Hall J. A., Mast M. Schmid (ed.), Cambridge University Press, Cambridge MA, 270-286.
Stressful First Impressions in Job Interviews
Finnerty Ailbhe N., Muralidhar Skanda, Nguyen Laurent Son, Pianesi Fabio, Gatica-Perez Daniel (2016), Stressful First Impressions in Job Interviews, in Proc. ACM Int. Conf. on Multimodal Interaction, 325-332, ACM, Tokyo325-332.
Training on the Job: Behavioral Analysis of Job Interviews in Hospitality
Muralidhar Skanda, Nguyen Laurent Son, Frauendorfer Denise, Odobez Jean-Marc, Schmid Mast Marianne, Gatica-Perez Daniel (2016), Training on the Job: Behavioral Analysis of Job Interviews in Hospitality, in Proc. ACM Int. Conf. on Multimodal Interaction, 84-91, 84-91.
Emergent Power Hierarchies and Group Performance
Frauendorfer D., Schmid Mast M., Sanchez-Cortes D., Gatica-Perez D. (2015), Emergent Power Hierarchies and Group Performance, in International Journal of Psychology, 50(5), 392-396.
Happy and Agreeable? Multi-Label Classification of Impressions in Social Video
Chavez-Martinez G, Ruiz-Correa S., Gatica-Perez D. (2015), Happy and Agreeable? Multi-Label Classification of Impressions in Social Video, in Proceedings of the International Conference on Mobile and Ubiquitous Multimedia (MUM), Linz.
Head Nod Detection From a Full 3D Model
Chen Y., Yu Y., Odobez J.-M. (2015), Head Nod Detection From a Full 3D Model, in Proceedings IEEE International Conference on Computer Vision (ICCV) Workshops, Santiago.
I Would Hire You in a Minute: Thin Slices of Nonverbal Behavior in Job Interviews
Nguyen L. S., Gatica-Perez D. (2015), I Would Hire You in a Minute: Thin Slices of Nonverbal Behavior in Job Interviews, in In Proceedings ACM Int. Conference on Multimodal Interaction (ICMI), Seattle.
In the Mood for Vlog: Multimodal Inference in Conversational Social Video
Sanchez-Cortes D., Kumano S., Otsuka K., Gatica-Perez D. (2015), In the Mood for Vlog: Multimodal Inference in Conversational Social Video, in ACM Trans. on Interactive Intelligent Systems, 5(2), 1-24.
Let Your Body Speak: Communicative Cue Extraction on Natural Interaction using RGBD Data
Marcos-Ramiro A., Pizarro D., Marron-Romera M., Gatica-Perez D. (2015), Let Your Body Speak: Communicative Cue Extraction on Natural Interaction using RGBD Data, in IEEE Trans. on Multimedia, 17(10), 1721-1732.
Mindless Computing: Designing Technologies to Subtly Influence Behavior
Adams A.T., Costa J., Jung M. F., Choudhury T. (2015), Mindless Computing: Designing Technologies to Subtly Influence Behavior, in Proc. ACM International Joint Conference on Pervasive and Ubiquitous Computing (Ubicomp), Osaka.
Personality Trait Classification via Co-Occurrent Multiparty Multimodal Event Discovery
Okada S., Aran O., Gatica-Perez D. (2015), Personality Trait Classification via Co-Occurrent Multiparty Multimodal Event Discovery, in Proc. ACM Int. Conf. on Multimodal Interaction (ICMI), Seattle.
Reliability and Validity of Nonverbal Thin Slices in Social Interactions
Murphy N., Hall J. A., Schmid Mast M., Ruben M. A., Frauendrofer D., Blanch-Hartigan D., Roter D., Nguyen L. (2015), Reliability and Validity of Nonverbal Thin Slices in Social Interactions, in Personality and Social Psychology Bulletin, 41(2), 199-213.
Signal Processing in the Workplace
Gatica-Perez D. (2015), Signal Processing in the Workplace, in IEEE Signal Processing Magazine, 32(1), 121-125.
Smiling in a Job Interview: When Less is More
Ruben M. A., Hall J. A., Schmid Mast M. (2015), Smiling in a Job Interview: When Less is More, in The Journal of Social Psychology, 155(2), 107-126.
Social Sensing for Psychology: Automated Interpersonal Behavior Assessment
Schmid Mast M., Gatica-Perez D., Frauendorfer D., Nguyen L. S., Choudhury T. (2015), Social Sensing for Psychology: Automated Interpersonal Behavior Assessment, in Current Directions in Psychological Science, 24(2), 154-160.
The Impact of Nonverbal Behavior in the Job Interview
Frauendorfer D., Schmid Mast M. (2015), The Impact of Nonverbal Behavior in the Job Interview, 220-247.
To Include or not to Include? Accuracy of Personality Judgements from Resumes with and without Photographs
Frauendorfer D., Schmid Mast M., Sutter C. (2015), To Include or not to Include? Accuracy of Personality Judgements from Resumes with and without Photographs, in Swiss Journal of Psychology, 74, 207-215.
Automatic Blinking Detection towards Stress Discovery
Marcos-Ramiro A., Pizarro-Perez D., Marron-Romera M., Gatica-Perez D. (2014), Automatic Blinking Detection towards Stress Discovery, in Proceedings of the ACM International Conference on Multimodal Interaction (ICMI), Istanbul.
Capturing Upper Body Motion in Conversation: an Appearance Quasi-Invariant Approach
Marcos-Ramiro A., Pizarro-Perez D., Marron-Romera M., Gatica-Perez D. (2014), Capturing Upper Body Motion in Conversation: an Appearance Quasi-Invariant Approach, in Proceedings of the ACM International Conference on Multimodal Interaction (ICMI), Istanbul.
Hire Me: Computational Inference of Hirability in Employment Interviews Based on Nonverbal Behavior
Nguyen L. S., Frauendorfer D., Schmid Mast M., Gatica-Perez D. (2014), Hire Me: Computational Inference of Hirability in Employment Interviews Based on Nonverbal Behavior, in IEEE Transactions on Multimedia, 16(4), 1018-1031.
Nonverbal communication and the vertical dimension of social relations
Hall J. A., Latu I. M., Carney D., Schmid Mast M. (2014), Nonverbal communication and the vertical dimension of social relations, in Tracy J. L., Anderson C., Cheng J. T. (ed.), 325-346.
Nonverbal Social Sensing in Action: Unobtrusive Recording and Extracting of Nonverbal Behavior in Social Interactions Illustrated with a Research Example
Frauendorfer D., Schmid Mast M., Nguyen L., Gatica-Perez D. (2014), Nonverbal Social Sensing in Action: Unobtrusive Recording and Extracting of Nonverbal Behavior in Social Interactions Illustrated with a Research Example, in Journal of Nonverbal Behavior, 38(2), 231-245.
Towards Personal Stress Informatics: Comparing Minimally Invasive Techniques for Measuring Daily Stress in the Wild
Adams P., Rahman T., Rabbi M., Voida S., Gay G., Choudhury T. (2014), Towards Personal Stress Informatics: Comparing Minimally Invasive Techniques for Measuring Daily Stress in the Wild, in Conference on Pervasive Computing technologies for Healthcare, Oldenburg.
The vertical dimension of social signaling
Schmid Mast Marianne, Hall J. A., The vertical dimension of social signaling, in Pantic M., Magnenat-Thlmann N., Burgoon J. K., Vinciarelli A. (ed.).
Understanding Small Groups
Gatica-Perez D., Aran O., Jayagopi D. B., Understanding Small Groups, in Pantic M., Magnenat-Thalmann N., Burgoon J., Vinciarelli A. (ed.), Cambridge University Press, Cambridge.

Collaboration

Group / person Country
Types of collaboration
Prof. Dr. Nora Murphy United States of America (North America)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Prof. Marta Marron, University of Alcala Spain (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
- Exchange of personnel
Speech, Music and Hearing group, KTH, Stockholm Sweden (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Dr. Fabio Pianesi, FBK-University of Trento Italy (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Exchange of personnel
Dr. Salvador Ruiz Correa, IPICYT Mexico (North America)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Dr Kazuhiro Otsuka, NTT Japan (Asia)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Vatel International School of Hospitality and Tourism Management Switzerland (Europe)
- Industry/business/other use-inspired collaboration
Prof. Matthias Mehl, University of Arizona United States of America (North America)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Prof. Amy Cuddy, Harvard University United States of America (North America)
- in-depth/constructive exchanges on approaches, methods or results
Prof. Melissa Ferguson, Cornell University United States of America (North America)
- in-depth/constructive exchanges on approaches, methods or results
Prof. Dr. Judith Hall United States of America (North America)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
Prof. Shogo Okada, Tokyo Institute of Technology Japan (Asia)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
- Exchange of personnel
Prof. Gillian Sandstrom, University of Essex Great Britain and Northern Ireland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
Yale University Individual talk Invited talk: Making the tracking of mental health as easy as tracking steps 27.11.2017 New Haven, United States of America Choudhury Tanzeem;
University of Zurich Individual talk Invited short course: Introduction to Smartphone and Social Sensing to Collect Behavioral Data in Psychological Science 02.11.2017 Zurich, Switzerland Gatica-Perez Daniel;
International Conference on Affective Computing and Intelligent Interaction Talk given at a conference Invited talk: Making the tracking of mental health as easy as tracking steps 26.10.2017 San Antonio TX, United States of America Choudhury Tanzeem;
CVPR Deep Affective Learning and Context Modeling Workshop Talk given at a conference Invited talk: Tracking behavioral symptoms of mental health and delivering personalized interventions using mobile and wearable devices 26.07.2017 Honolulu, Hawaii, United States of America Choudhury Tanzeem;
University of Lausanne Individual talk Invited talk: Images, Tags, and Interpretable Machine Learning 20.06.2017 Lausanne, Switzerland Gatica-Perez Daniel;
ACM Mobisys Women's Workshop Talk given at a conference Invited talk: Tracking behavioral symptoms of mental health and delivering personalized interventions using mobile and wearable devices 19.06.2017 Niagara Falls NY, United States of America Choudhury Tanzeem;
Carnegie Mellon University HCII Seminar Individual talk Invited talk: Tracking behavioral symptoms of mental health and delivering personalized interventions using mobile and wearable devices 02.06.2017 Pittsburgh, PA, United States of America Choudhury Tanzeem;
Workshop on Multi-Modal Analyses For Enabling More Socially Aware Agents Individual talk Robust Extraction of Non-Verbal Behaviours from Video Sensors 01.01.2017 KTH, Stockholm, Sweden Odobez Jean-Marc;
L'Innovation en Question Talk given at a conference Invited talk: Computing for and with People 16.11.2016 Sierre, Switzerland Gatica-Perez Daniel;
Radboud University Individual talk Invited talk: Analyzing Job Interviews: A Multimodal Approach 10.10.2016 Nijmegen, Netherlands Gatica-Perez Daniel;
ECCV Workshop on Apparent Personality Analysis Talk given at a conference Invited talk: Analyzing Job Interviews: A Multimodal Approach 09.10.2016 Amsterdam, Netherlands Gatica-Perez Daniel;
Second World Conference on Personality Talk given at a conference What you do, tells me who you are - immediately: Social sensing for personality psychology 01.05.2016 Buzios, Brazil Schmid Mast Marianne;
Gästekolloquium Arbeits- und Organisationspsychologie der Universität Saarbrücken Individual talk Social Sensing und Immersion in der virtuellen Realität ermöglichen automatisches Verhaltensfeedback für Personalauswahl und Personalentwicklung 08.04.2016 Saarbrücken, Germany Schmid Mast Marianne; Frauendorfer Denise; Nguyen Laurent;
Preconference New Methods of the Conference of the Society for Personality and Social Psychology Talk given at a conference Automated interpersonal behavioral assessment 20.01.2016 San Diego, United States of America Frauendorfer Denise; Schmid Mast Marianne;
17th conference of hte Society for Social ans Personality Psychology Talk given at a conference Can applicant nonverbal vehavior predict future job performance? 20.01.2016 San Diego, United States of America Schmid Mast Marianne; Frauendorfer Denise;
Institute of Neuroscience and Psychology seminar Individual talk Robust Non-verbal behavior sensing for human interaction analysis 04.01.2016 University of Glasgow, Great Britain and Northern Ireland Odobez Jean-Marc;
ACM International Workshop on Advancements in Social Signal Processing for Multimodal Interaction Talk given at a conference Invited talk: Crowds Watching Crowds: Impressions and Behavior in Social Video 13.11.2015 Seattle, United States of America Gatica-Perez Daniel;
3TU.H&T/SIKS Autumn School on Humans and Technology Individual talk Invited talk: Crowds Watching Crowds: Impressions and Behavior in Social Video 20.10.2015 Zeist, Netherlands Gatica-Perez Daniel;
Workshop for the Conference Universitaire de Suisse Occidentale (CUSO) Talk given at a conference Video Analysis: Coding of Observable Behavior 01.10.2015 Neuchatel, Switzerland Frauendorfer Denise;
KTH Speech, Music, and Hearing Seminar Individual talk Computational Analysis Of Behavior In Employment Interviews And Video Resumes 17.09.2015 Stockholm, Sweden Nguyen Laurent;
NTT Communication Science Labs Individual talk Invited talk: Sensing and Analyzing Employment Interviews: A Multimodal Approach 14.09.2015 Atsugi, Japan Gatica-Perez Daniel;
International Workshop on Human Behavior Understanding Talk given at a conference Invited talk: Tracking behavioral symptoms of mental illness and delivering personalized interventions using smartphones and wearables 08.09.2015 Osaka, Japan Choudhury Tanzeem;
14th Conference of the Swiss Society of Psychology Talk given at a conference Meta-Accuracy of Applicants’ Interview Performance 08.09.2015 Geneva, Switzerland Schmid Mast Marianne; Frauendorfer Denise;
7th IJHR-Conference Section Romande Talk given at a conference Invited talk: Hiérarchie et pouvoir au travail 08.09.2015 Lausanne, Switzerland Schmid Mast Marianne;
SMART School on Computational, Social, and Behavioral Sciences Individual talk Invited talk: Understanding conversational behavior in social video 31.08.2015 Paris, France Gatica-Perez Daniel;
International Society for Bipolar Disorder 2015 Annual Conference Talk given at a conference Invited talk: Tracking behavioral symptoms of bipolar disorder using automated sensing and delivering personalized interventions using smartphones 06.06.2015 Toronto, Canada Choudhury Tanzeem;
CS Colloquium, Johns Hopkins University Individual talk Tracking behavioral symptoms of mental illness and delivering personalized interventions using smartphones and wearables 05.05.2015 Baltimore, United States of America Choudhury Tanzeem;
NAE-AAES Convocation of Engineering Professional Societies Individual talk Tracking Behavioral Symptoms of Mental Illness Using Automated Sensing and Delivering Personalized Interventions Using Smartphones 20.04.2015 Virginia, United States of America Choudhury Tanzeem;
16th Conference of the Society for Social and Personality Psychology Talk given at a conference The Social Chameleon: The Link between Interpersonal Accuracy and Behavioral Adaptability 26.02.2015 Long Beach , United States of America Schmid Mast Marianne; Frauendorfer Denise;
Int. Workshop on Multi-Party Real-World Human-Robot Interaction Talk given at a conference Invited talk: Social inference from nonverbal cues, small and big 16.11.2014 Istanbul, Turkey Gatica-Perez Daniel;
General Meeting of the European Association of Social Psychology Poster Applicatn verbal immediacy in the job interview: Beneficial or detrimental for the job interview outcome? 20.09.2014 Amsterdam, Netherlands Frauendorfer Denise; Schmid Mast Marianne;
Int. Workshop on Human Behavior Understanding Talk given at a conference Invited talk: When the crowd watches the crowd: understanding conversational social video 12.09.2014 Zurich, Switzerland Gatica-Perez Daniel;
CITEC (Cluster of Excellence Cognitive Interaction Technology) seminar Individual talk Attention recognition: from contextual analysis of head poses to 3D gaze tracking using remote RGB-D sensors 20.08.2014 Bielefeld University, Germany Odobez Jean-Marc;
Interpersonal Accuracy Workshop Individual talk Invited talk: Interpersonal accuracy in personnel selection 07.05.2014 Neuchatel, Switzerland Frauendorfer Denise;
15th conference of the Society for Social and Personality Psychology Poster Interpersonal sensitivity and adequate interpersonal behavior: Empirical evidence of an existing link for women 13.02.2014 Austin, United States of America Schmid Mast Marianne; Frauendorfer Denise;
3rd Nonverbal Preconference of the Society for Personality and Social Psychology Talk given at a conference Applicant nonverbal and verbal immediacy in the job interview: Same but different? 20.01.2014 Austin, United States of America Schmid Mast Marianne; Frauendorfer Denise;
Conference of the German Assiciation of Psychology Talk given at a conference In nur 40 sekunden: Korrekte Voraussagen künftiger Arbeitsleistung auf Basis kurzer Videosequenzen des Bewerbungsintervies 20.01.2014 Bochum, Germany Frauendorfer Denise;
15th Conference of the Society for Personality and Social Psychology Poster The structure of personality judgment accuracy 20.01.2014 Austin, United States of America Frauendorfer Denise; Schmid Mast Marianne;


Knowledge transfer events

Active participation

Title Type of contribution Date Place Persons involved
General Annual Meeting of the Society for Swiss Assessment Performances, exhibitions (e.g. for education institutions) 25.03.2017 Bern, Switzerland Schmid Mast Marianne;


Self-organised

Title Date Place
Workshop on Ubiquitous First Impressions and Ubiquitous Awareness 23.11.2017 Martigny, Switzerland
Workshop on Novel Technologies to Measure Verbal and Nonverbal Behavior 08.09.2015 Lausanne, Switzerland
7th IJHR-Conference Section Romande 08.09.2015 Lausanne, Switzerland

Communication with the public

Communication Title Media Place Year
Video/Film Langage non verbal: conseils pour les séances en groupe Western Switzerland 2017
Video/Film Langage nonverbal: conseils pour l'entretien d'embauche Western Switzerland 2017
Media relations: print media, online media Et si l'entretien d'embauche était un leurre? Bilan Western Switzerland 2016
Media relations: print media, online media Comment déceler et utiliser vos soft skills Bilan Magazine Western Switzerland 2015
Media relations: radio, television Comportements humains: l’Idiap sonde les smartphones Canal 9 Valais Western Switzerland 2015
Media relations: print media, online media On ne peut pas mentir à l'ordinateur Bilan Western Switzerland 2015
Media relations: print media, online media Watching me, watching you: Making sense of social interaction Blog sur HECImpact International 2015
Media relations: print media, online media Watching me, watching you: making sense of social interaction HRZone International 2015

Awards

Title Year
Listed as one of the 50 most influential psychologists alive 2018
ACM Int. Conf. on Multimodal Interaction (ICMI) Community Service Award 2017
Best Paper Award , ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing (Ubicomp) 2016
Doctoral Student Paper Award 2014, Idiap Research Institute 2014

Associated projects

Number Title Start Funding scheme
127542 SONVB: Sensing and Analyzing Organizational Nonverbal Behavior 01.06.2010 Sinergia
127542 SONVB: Sensing and Analyzing Organizational Nonverbal Behavior 01.06.2010 Sinergia
191634 The Nature of Artificial Intelligence 01.12.2020 Agora

Abstract

First impressions matter. Brief interpersonal interactions with people we do not know occur everywhere and often, and the evaluations we make of others (and that others make of us) can have profound effects. When we meet somebody for the first time, we form an impression very quickly based on their nonverbal behavior (tone of voice, speech modulation, speech duration, gaze, facial expressions, body movements and postures) and the spoken words, and this impression in turn guides our behavior. Specifically, (first) impressions in the workplace can affect outcomes like being hired or promoted, and positive impressions are critical in entire sectors of the economy, including sales, service, and hospitality, where employees are frequently evaluated, rewarded, and advanced in their careers based on the impressions they convey. Customers judge quality of service based on their impressions about the service provider -- how friendly or competent or helpful.Research in organizational psychology and nonverbal communication has revealed some of the connections existing between nonverbal behavior and impressions in the workplace for some years, including links between immediacy behavior and hiring decisions or ratings of supervisors. However, and despite the fact that impressions at work are ubiquitous, much of the existing research in this domain has largely been done in the laboratory, for single organizational situations, and based on single interactions. Furthermore, one of the fundamental goals of organizational behavior research -- how to make these findings useful for training and improvement of skills by employees -- has often been disconnected from laboratory experiments given the lack of methods and tools to systematically study favorable first impressions and other related variables in the field, with multiple job-related situations, over time, and embedded with the training process.This is what UBImpressed aims to achieve through a multidisciplinary research approach involving academics in Work and Organizational Psychology, Multimodal Signal Processing, Ubiquitous Computing, and Social Computing. The project innovatively integrates nonverbal communication research with mobile sensing, perceptual computing, machine learning, and mobile visualization applications to understand (1) what nonverbal behaviors are related to conveying favorable first impressions in different domains within the organizational context; (2) which of these impression-related behavioral cues can be robustly extracted and analyzed by automatic means over multiple physical settings; and (3) how favorable first impressions in the workplace can be trained by integrating expert knowledge with automatic sensing, analysis, and visualization technology.The project will advance the state of the art regarding nonverbal communication and organizational psychology and automated analysis of human behavior, and brings about new research directions that have the potential to impact jointly in psychology and computer science. These include: the partnership with a real organization in the hospitality industry for whose members favorable first impressions are critical for success; the ecologically valid analysis of first impressions in a population for an extended period of time and under psychological and computational lenses, both in the laboratory and the field; the investigation of online video to generate first-impression data, whose analysis could be transferred to the physical world; the design of new machine perception and machine learning methodologies to extract behavioral features, discover behavioral patterns in individuals, and make automatic predictions about first impressions; the embedding of data-driven machine analysis and human expertise in training into real applications to be used by people to improve their skills; and the generation of research resources that have the potential of being widely reused by the research community.
-