Flaky Test Detection and Automated Root Cause Analysis - Automated Software Testing

Overview

Another expertise of mine is to provide support to developers and colleagues by reducing flaky tests: a test that exhibits both passing and failing outcomes when no changes are introduced into the codebase. The flaky test can be in the test suite, product, or CI infrastructure. Regression testing, automatic or manual, is intended to ensure that changes made in any part of the system do not break existing functionality. Developers submit code changes with the expectation that test failures will be associated with the code modifications. Unfortunately, rather than being the result of changes to the code, some test failures occur due to flaky tests. I have developed a tool [4] that uses a multifactor strategy to identify test flakiness in automated testing in CI/CD and provide automated root cause analysis explaining why the test cases are flaky, thus increasing confidence in the test suite and final product. My published research using machine learning to improve automated test case activities can on test flakiness can be online.