An Evaluation of Machine Learning Methods forPredicting Flaky Tests

Abstract

In this paper we have investigated as a means of prevention the feasibility of using machine learning (ML) classifers for flaky test prediction in project written with Python. This study compares the predictive accuracy of the three machine learning classifers (Naive Bayes, Support Vector Machines, and Random Forests) with each other. We compared our fndings with the earlier investigation of similar ML classifers for projects written inJava. Authors in this study investigated if test smellsare good predictors of test fakiness. As developers need to trust the predictions of ML classifers, they wish to know whichtypes of input data or test smells cause more false negatives and false positives. We concluded that RF performed better whenit comes to precision (> 90%) but provided very low recall (< 10%) as compared to NB (i.e., precision < 70% and recall >30%)and SVM (i.e., precision < 70% and recall >60%)

Azeem Ahmad

PhD Candidate in Software Engineering

The focus of my research is to use machine learning and AI methods to solve real-time problems related to continuous integration and delivery (CI/CD), software test optimization, and data visualization in CI/CD in European Industries (i.e., https://www.software-center.se/partners/) that are part of the software center (www.software-center.se). I investigate and optimize the Continuous Integration and Delivery pipeline and how to visualize data streams, currently flowing in the pipeline. In addition to this, a part of my research is to optimize software testing and quality.