Project

Back to overview

G-DEDUCE: Gestures in the Design of new tools to express Emotions During User Centred Evaluations

Applicant Landoni Monica
Number 138151
Funding scheme Project funding (Div. I-III)
Research institution Istituto del Software (SI) Facoltà di scienze informatiche
Institution of higher education Università della Svizzera italiana - USI
Main discipline Information Technology
Start/End 01.01.2012 - 31.12.2012
Approved amount 55'852.00
Show all

All Disciplines (2)

Discipline
Information Technology
Psychology

Keywords (6)

Affective Computing; User studies; Evaluation methodology; Gestures recognition; Tools for Evaluators; Emotions

Lay Summary (English)

Lead
Lay summary

In recent years the presence and use of interactive devices has been growing rapidly. Mobile phones are omnipresent and smart phones keep us connected to social networks at all time, cars are equipped with satellite navigation systems and can sense when we are tired, and children’s stuffed animals and dolls can ‘talk’ and express wishes. With interactive devices becoming ever more ubiquitous, the field of human-computer interaction has seen a turn from investigating business environments and how to make interaction more effective and efficient towards finding out how interaction can happen in a more joyous and satisfying way in all facets of everyday life. We find ourselves in need of measures and tools to assess these experiences.  We are proposing G-DEDUCE to extend project DEDUCE, funded by SNSF since January 2009, and expand its findings while maintaining the same aim and focus.

The aim of this project is to find new tools that allow assessing users’ affective states in evaluation trials of interactive products.

 In a series of user studies run by DEDUCE, we have observed children playing video games and could see how they expressed their emotional states in a non-verbal way. When we watched recordings with the sound turned off, we could still easily identify when children were happy and proud after winning a game, or disappointed and sad after loosing only from observing how they moved around.

Literature suggests that reading emotional states of the people around us comes natural to us. As we are social beings it is important to know whether our environment is friendly and at ease or tense and hostile. Non-verbal expression of emotion involves the tone of the voice, facial expressions, and expressions from the rest of the body, i.e. gesture, posture and body movements.

Here we propose to investigate and develop an automatic emotion recognition system based on posture and body movements.

We will develop a series of working prototypes for gesture capture and interpretation. We will then build a database of affective movement data, by recording movement data and coding the recordings by walkthroughs with participants and/or by observer ratings, finally we will train a classifier to automatically identify emotional states.

The originality of the study is in its attention to a crucial area of interdisciplinary research that is evaluation techniques and their use in different scenarios. The project will deliver interactive tools to be used both at evaluation time and when analysing data by providing an initial, if rough, description of user interactions with system ready for interpretation.

Direct link to Lay Summary Last update: 21.02.2013

Responsible applicant and co-applicants

Employees

Name Institute

Publications

Publication
Assessing Gaming Experience through Bodily Expression of Affect
Pasch M, Landoni Monica (2012), Assessing Gaming Experience through Bodily Expression of Affect, in Fun and Games 2012, Toulouse France 90(1-2), 90(1-2).
Building Corpora of Bodily Expressions of Affect
Pasch Marco, Landoni Monica (2012), Building Corpora of Bodily Expressions of Affect, in Measuring Behaviour 2012, Utrech.
Using gamers’ movements to assess their affective experience in movement-based games.
Pasch Marco, Landoni Monica (2012), Using gamers’ movements to assess their affective experience in movement-based games., in Fun and Games 2012.
Longitudinal assessment of a user experience evaluation tool for children
Pasch Marco, Landoni Monica (2012), Longitudinal assessment of a user experience evaluation tool for children, in CHI 2012 Workshop: Theories, methods and case studies of longitudinal HCI research..

Collaboration

Group / person Country
Types of collaboration
Department of Computing, Goldsmiths Great Britain and Northern Ireland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
UCL INTERACTION CENTRE Division of Psychology & Language Sciences and Department of Computer Science Great Britain and Northern Ireland (Europe)
- in-depth/constructive exchanges on approaches, methods or results

Associated projects

Number Title Start Funding scheme
120139 DEDUCE: Design of new tools to express Emotions During User Centred Evaluations 01.01.2009 Project funding (Div. I-III)
120139 DEDUCE: Design of new tools to express Emotions During User Centred Evaluations 01.01.2009 Project funding (Div. I-III)

Abstract

This will be an extension to existing project DEDUCE, as such it will aim at designing, implementing and evaluating new tools for allowing easier interactions with users when measuring usability of interactive systems.When developing new tools for supporting evaluation involving adults these should focus on the role of capturing emotions, negative or positive, generated by interacting with system or simply by its existence.This type of tools is expected to be particularly suitable for supporting designers during iterative sessions allowing for fast informal feedback to be gathered and inform next stage of design.It is also expected that the same tools could be used to facilitate a number of human computer interactions that would normally involve the use of linguistic skills. The originality of the study is in its attention to a crucial area of interdisciplinary research often over-looked, that is evaluation techniques and their use in different scenarios. This extension will enable us to explore further the way body gesture could be captured, classified and interpreted as a direct and effective form of users' feedback.
-