Project

Back to overview

Increasing the effectiveness of usability testing: the role of testing method, cultural background and system features beyond usability

English title Increasing the effectiveness of usability testing: the role of testing method, cultural background and system features beyond usability
Applicant Sauer Jürgen
Number 140359
Funding scheme Project funding
Research institution Departement für Psychologie Universität Freiburg
Institution of higher education University of Fribourg - FR
Main discipline Psychology
Start/End 01.07.2012 - 31.12.2016
Approved amount 377'344.00
Show all

Keywords (1)

Usability test, cultural background, non-visual ae

Lay Summary (English)

Lead
Lay summary

Usability testing is a method used in industry that examines how well consumer products (e.g., kettle, mobile phone) can be operated by future customers before the products are launched on the market. By modelling typical situations in which users operate consumer products, a usability test allows the identification of problems users encounter during product operation. Against this background, the present project aims to examine a range of factors that are suspected to have an undue influence on the results of usability tests. This includes methodological questions such as the location in which such a usability test takes place (e.g. should a mobile phone be tested in a well controllable usability laboratory or in a more natural environment such as a coffee bar?). A further methodological issue to be addressed is whether a one-off test is sufficient (e.g. users operates a new mobile phone for two hours) or multiple-testing sessions are necessary (e.g., users operates a new mobile phone over a period of several weeks). The project also examines the impact of cultural background on the outcomes of usability tests. For example, we are interested in the influence of the cultural match (or mismatch) between the user being tested and the person running the usability test. Finally, the project examines the question whether a high non-visual aesthetic appeal of a consumer product (e.g., a mobile phone feels very nice when being touched) will also influence the way the product is being operated and perceived (e.g., is it also considered to be more usable and effective?). In addition to the scientific implications of the work, the results of the project are also expected to provide benefits to usability practitioners in that it makes them more aware of undue influences of certain testing conditions in the completion of usability tests.

Direct link to Lay Summary Last update: 21.02.2013

Responsible applicant and co-applicants

Employees

Publications

Publication
The role of non-visual aesthetics in consumer product evaluation
Sonderegger Andreas, Sauer Juergen (2015), The role of non-visual aesthetics in consumer product evaluation, in International Journal of Human-Computer Studies, 84, 19-32.
Extra-laboratorial usability tests: an empirical comparison of remote and classical field testing with lab testing
SauerJuergen, SondereggerAndreas, Heyden Klaus, BillerJasmin, KlotzJulia, UebelbacherAndreas, Extra-laboratorial usability tests: an empirical comparison of remote and classical field testing with lab testing, in Applied Ergonomics.
No need to read: Developing a pictorial single-item scale for measuring perceived usability.
Baumgartner Juergen, Sonderegger Andreas, Sauer Juergen, No need to read: Developing a pictorial single-item scale for measuring perceived usability., in International Journal of Human-Computer Studies.
The influence of cultural background of test participants and test facilitators in online product evaluation.
SauerJuergen, SondereggerAndreas, Hoyos Alvarez Mariana, The influence of cultural background of test participants and test facilitators in online product evaluation., in International Journal of Human-Computer Studies..

Collaboration

Group / person Country
Types of collaboration
Prof. Kai-Christoph Hamborg, University of Osnabrück Germany (Europe)
- in-depth/constructive exchanges on approaches, methods or results
Dr Daniel Felix, ergonomie & technologie (e&t) GmbH, Zurich Switzerland (Europe)
- Industry/business/other use-inspired collaboration

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
Proceedings of the 32nd annual ACM conference on Human factors in computing systems (CHI 2014), pp Talk given at a conference The influence of aesthetics in usability testing: the case of dual-domain products 26.04.2014 Toronto, Canada Sonderegger Andreas; Sauer Jürgen;
13. Biannual Congress of the Swiss Psychological Society Talk given at a conference Prototype aesthetics in usability tests - the influence of acoustic, haptic and visual attractiveness on test outcomes.a 11.09.2013 Basel, Switzerland Sauer Jürgen; Sonderegger Andreas;
UbiComp2013, Atelier of Smart Garments and Accessories Talk given at a conference Smart Garments - the Issue of Usability and Aesthetics. 09.09.2013 Zürich, Switzerland Sonderegger Andreas;


Associated projects

Number Title Start Funding scheme
122490 The utility of usability tests: An examination of factors that influence test outcomes 01.12.2008 Project funding

Abstract

Extending the work of a preceding SNSF-project, the current project proposal focuses on the question of how the utility of usability tests can further be improved. While there is little doubt that usability testing is an effective product evaluation method, researchers and practitioners have identified a number of factors surrounding the testing context, which may have an undesirable impact on test outcomes and hence impair the effectiveness of usability tests. Therefore, these contextual factors are to be empirically examined and the degree of their influence is to be determined. The present research project is guided by the Four-Factor Framework of Contextual Fidelity, which proposes a number of aspects that are suspected to have such an undue influence on test outcomes. The proposed project comprises a total of eight experiments, focusing on three main areas. The first area (3 experiments) is concerned with methodological issues of usability testing. This work addresses the impact of using traditional testing approaches (e.g., lab-based testing and single-session testing) on usability test outcomes by comparing these with the more realistic testing conditions of field-based testing and of longitudinal approaches involving multi-session testing. Furthermore, this work includes the empirical evaluation of remote testing as a methodological approach adopted more recently in usability testing. The second area (2 experiments) addresses the influence of cultural background on the outcomes of usability tests. Given that many products are developed for international markets, there is an increasing need to better understand the implications of carrying out usability tests in different cultures. A systematic empirical evaluation of usability test outcomes across several cultures will provide evidence for the influence of intercultural differences. The third area (3 experiments) is concerned with the influence of product features that are unrelated to product usability but still appear to have an influence on perceived usability and sometimes even on objective measures of usability. Examples of such product characteristics include product brand but also visual and non-visual aesthetic features. The proposed project is expected to create benefits at several levels. First, it will provide practitioners with recommendations about important issues to be considered during usability testing, notably concerning the utility of various methodological approaches. Second, the research community will benefit from the empirical data generated by the planned experiments since it allows a more precise assessment of the degree of influence of each of the factors being examined.
-