A Multi-factor strategy for flaky test detection and automated root cause analysis

Abstract

Developers often spend time to determine whether test case failures are real failures or flaky. The flaky tests, also known as non-deterministic tests, switch their outcomes without any modification in the codebase. Flaky tests in a test suite reduce the confidence of developers in the test suite as well as in the quality of a product. The most common approach to detect flaky tests is re-running test cases which is resource consuming and unreliable. Furthermore, re-running only indicates what tests are flaky but does not reveal the root causes of test flakiness.

Azeem Ahmad

PhD Candidate in Software Engineering

The focus of my research is to use machine learning and AI methods to solve real-time problems related to continuous integration and delivery (CI/CD), software test optimization, and data visualization in CI/CD in European Industries (i.e., https://www.software-center.se/partners/) that are part of the software center (www.software-center.se). I investigate and optimize the Continuous Integration and Delivery pipeline and how to visualize data streams, currently flowing in the pipeline. In addition to this, a part of my research is to optimize software testing and quality.