Project

Back to overview

The utility of usability tests: An examination of factors that influence test outcomes

English title The utility of usability tests: An examination of factors that influence test outcomes
Applicant Sauer Jürgen
Number 122490
Funding scheme Project funding (Div. I-III)
Research institution Departement für Psychologie Universität Freiburg
Institution of higher education University of Fribourg - FR
Main discipline Psychology
Start/End 01.12.2008 - 29.02.2012
Approved amount 203'323.00
Show all

Keywords (8)

usability; ergonomics; performance; Usability test; perceived usability; emotion; aesthetics; prototype

Lay Summary (English)

Lead
Lay summary
Usability tests are commonly used in industry to identify shortcomings and weaknesses in consumer product design before the product is being launched on the market. A usability test models the usage situation, in which users are asked to carry out a set of tasks with a consumer product (e.g., coffee machine) to see what problems the user has encountered during product operation. Previous research has indicated remarkable inconsistencies across usability tests with regard to the usability problems being identified. The present project aims to identify factors that have an influence on test outcomes with a view to increase the effectiveness and utility of usability tests. Factors that are expected to have an influence are, for example, presence of observers, product aesthetics, and the kind of prototype used (e.g., paper prototype, computer simulation). In a series of experiments carried out in a usability laboratory but also in people’s homes, the influence of these factors is tested by comparing several conditions (e.g., one group of users operates an attractively designed prototype while the other group operates an unattractive one). The research is carried out with a range of typical consumer products, such as mobile phone, digital camera and coffee machines. This also allows us to determine the generalisability of the effects across a certain type of product (e.g., products with high and low prestige value). Finally, the project aims to examine the transferability of the findings from the domain of consumer product to a work context (e.g., is the influence of aesthetics on perceived product usability lower when a mobile phone is used at work than at home?). On the basis of the findings of the project, we will be able to provide some recommendations to practitioners involved in usability testing about how they can improve their procedure to conduct a usability test.
Direct link to Lay Summary Last update: 21.02.2013

Responsible applicant and co-applicants

Employees

Publications

Publication
Effects of perceived prototype fidelity in usability testing under different conditions of observer presence
Uebelbacher Andreas, Sonderegger Andreas, Sauer Juergen (2013), Effects of perceived prototype fidelity in usability testing under different conditions of observer presence, in Interacting with Computers, 25, 91-101.
The influence of product aesthetics and usability over the course of time: a longitudinal field experiment.
Sonderegger Andreas, Zbinden Gerold, Uebelbacher Andreas, Sauer Juergen (2012), The influence of product aesthetics and usability over the course of time: a longitudinal field experiment., in Ergonomics, 55, 840-853.

Collaboration

Group / person Country
Types of collaboration
Kai-Christoph Hamborg Germany (Europe)
- in-depth/constructive exchanges on approaches, methods or results

Communication with the public

Communication Title Media Place Year
Talks/events/exhibitions  Uebelbacher, A. (2011). Die Königsmethode im Usability Engineering: Aktuelle Forschungsergebnisse Western Switzerland 2011

Associated projects

Number Title Start Funding scheme
140359 Increasing the effectiveness of usability testing: the role of testing method, cultural background and system features beyond usability 01.07.2012 Project funding (Div. I-III)

Abstract

The present project is concerned with the utility of usability tests and how their effectiveness can be improved. While usability tests are undoubtedly helpful in identifying weaknesses in product design, there is little systematic research work examining factors that influence test outcomes. A better understanding of the influence of these factors would allow us to increase the effectiveness of usability tests. There are a number of factors that may impair the reliability and validity of usability tests, ranging from the presence of test observers to the kind of prototype being used. We have developed a model, termed the Four-Factor Framework of Contextual Fidelity, to determine the extent to which the outcomes of usability tests are influenced by various elements of the testing situation. This Four-Factor Framework of Contextual Fidelity guides the proposed research programme, which comprises a series of experiments that evaluate the influence of the four general factors (user, environment, task, technical system). Of particular interest are the elements prototype fidelity, product aesthetics and observer presence for the outcome of usability tests. The research is carried out with typical interactive consumer products (e.g., mobile phone, digital camera), including replications of test scenarios with a product to determine the generalisability of the findings. Furthermore, we plan to compare the pattern of effects for summative and formative evaluation. Finally, we will examine the transferability of the findings from the domain of consumer products to a work context. The findings of the research project will provide guidance to designers and usability practitioners about issues to be considered when conducting usability tests.
-