Back to overview

Hyperparameter Optimization to Improve Bug Prediction Accuracy

Type of publication Peer-reviewed
Publikationsform Proceedings (peer-reviewed)
Author Osman Haidar, Ghafari Mohammad, Nierstrasz Oscar,
Project Agile Software Analysis
Show all

Proceedings (peer-reviewed)

Page(s) 33 - 38
Title of proceedings 1st International Workshop on Machine Learning Techniques for Software Quality Evaluation (MaLTeSQuE 2017)
DOI 10.1109/maltesque.2017.7882014

Open Access

Type of Open Access Website


Bug prediction is a technique that strives to identify where defects will appear in a software system. Bug prediction employs machine learning to predict defects in software entities based on software metrics. These machine learning models usually have adjustable parameters, called hyperparameters, that need to be tuned for the prediction problem at hand. However, most studies in the literature keep the model hyperparameters set to the default values provided by the used machine learning frameworks. In this paper we investigate whether optimizing the hyperparameters of a machine learning model improves its prediction power. We study two machine learning algorithms: k-nearest neighbours (IBK) and support vector machines (SVM). We carry out experiments on five open source Java systems. Our results show that (i) models differ in their sensitivity to their hyperparameters, (ii) tuning hyperparameters gives at least as accurate models for SVM and significantly more accurate models for IBK, and (iii) most of the default values are changed during the tuning phase. Based on these findings we recommend tuning hyperparameters as a necessary step before using a machine learning model in bug prediction.