Back to overview

How developers engage with static analysis tools in different contexts

Type of publication Peer-reviewed
Publikationsform Original article (peer-reviewed)
Author Vassallo Carmine, Panichella Sebastiano, Palomba Fabio, Proksch Sebastian, Gall Harald C., Zaidman Andy,
Project SURF-MobileAppsData
Show all

Original article (peer-reviewed)

Journal Empirical Software Engineering
Volume (Issue) 25(2)
Page(s) 1419 - 1457
Title of proceedings Empirical Software Engineering
DOI 10.1007/s10664-019-09750-5

Open Access

Type of Open Access Repository (Green Open Access)


Automatic static analysis tools (ASATs) are instruments that support code quality assessment by automatically detecting defects and design issues. Despite their popularity, they are characterized by (i) a high false positive rate and (ii) the low comprehensibility of the generated warnings. However, no prior studies have investigated the usage of ASATs in different development contexts (e.g., code reviews, regular development), nor how open source projects integrate ASATs into their workflows. These perspectives are paramount to improve the prioritization of the identified warnings. To shed light on the actual ASATs usage practices, in this paper we first survey 56 developers (66% from industry and 34% from open source projects) and interview 11 industrial experts leveraging ASATs in their workflow with the aim of understanding how they use ASATs in different contexts. Furthermore, to investigate how ASATs are being used in the workflows of open source projects, we manually inspect the contribution guidelines of 176 open-source systems and extract the ASATs’ configuration and build files from their corresponding GITHUB repositories. Our study highlights that (i) 71% of developers do pay attention to different warning categories depending on the development context; (ii) 63% of our respondents rely on specific factors (e.g., team policies and composition) when prioritizing warnings to fix during their programming; and (iii) 66% of the projects define how to use specific ASATs, but only 37% enforce their usage for new contributions. The perceived relevance of ASATs varies between different projects and domains, which is a sign that ASATs use is still not a common practice. In conclusion, this study confirms previous findings on the unwillingness of developers to configure ASATs and it emphasizes the necessity to improve existing strategies for the selection and prioritization of ASATs warnings that are shown to developers.