Project

Usability test, cultural background, non-visual ae

Lead
Lay summary
Usability testing is a method used in industry that examines how well consumer products (e.g., kettle, mobile phone) can be operated by future customers before the products are launched on the market. By modelling typical situations in which users operate consumer products, a usability test allows the identification of problems users encounter during product operation. Against this background, the present project aims to examine a range of factors that are suspected to have an undue influence on the results of usability tests. This includes methodological questions such as the location in which such a usability test takes place (e.g. should a mobile phone be tested in a well controllable usability laboratory or in a more natural environment such as a coffee bar?). A further methodological issue to be addressed is whether a one-off test is sufficient (e.g. users operates a new mobile phone for two hours) or multiple-testing sessions are necessary (e.g., users operates a new mobile phone over a period of several weeks). The project also examines the impact of cultural background on the outcomes of usability tests. For example, we are interested in the influence of the cultural match (or mismatch) between the user being tested and the person running the usability test. Finally, the project examines the question whether a high non-visual aesthetic appeal of a consumer product (e.g., a mobile phone feels very nice when being touched) will also influence the way the product is being operated and perceived (e.g., is it also considered to be more usable and effective?). In addition to the scientific implications of the work, the results of the project are also expected to provide benefits to usability practitioners in that it makes them more aware of undue influences of certain testing conditions in the completion of usability tests.

Direct link to Lay Summary

Last update: 21.02.2013

Active participation

Number	Title	Start	Funding scheme

Extending the work of a preceding SNSF-project, the current project proposal focuses on the question of how the utility of usability tests can further be improved. While there is little doubt that usability testing is an effective product evaluation method, researchers and practitioners have identified a number of factors surrounding the testing context, which may have an undesirable impact on test outcomes and hence impair the effectiveness of usability tests. Therefore, these contextual factors are to be empirically examined and the degree of their influence is to be determined. The present research project is guided by the Four-Factor Framework of Contextual Fidelity, which proposes a number of aspects that are suspected to have such an undue influence on test outcomes. The proposed project comprises a total of eight experiments, focusing on three main areas. The first area (3 experiments) is concerned with methodological issues of usability testing. This work addresses the impact of using traditional testing approaches (e.g., lab-based testing and single-session testing) on usability test outcomes by comparing these with the more realistic testing conditions of field-based testing and of longitudinal approaches involving multi-session testing. Furthermore, this work includes the empirical evaluation of remote testing as a methodological approach adopted more recently in usability testing. The second area (2 experiments) addresses the influence of cultural background on the outcomes of usability tests. Given that many products are developed for international markets, there is an increasing need to better understand the implications of carrying out usability tests in different cultures. A systematic empirical evaluation of usability test outcomes across several cultures will provide evidence for the influence of intercultural differences. The third area (3 experiments) is concerned with the influence of product features that are unrelated to product usability but still appear to have an influence on perceived usability and sometimes even on objective measures of usability. Examples of such product characteristics include product brand but also visual and non-visual aesthetic features. The proposed project is expected to create benefits at several levels. First, it will provide practitioners with recommendations about important issues to be considered during usability testing, notably concerning the utility of various methodological approaches. Second, the research community will benefit from the empirical data generated by the planned experiments since it allows a more precise assessment of the degree of influence of each of the factors being examined.