CS professor Darko Marinov and three of his former students, Qingzhou Luo, Farah Hariri, and Lamyaa Eloussi, were awarded an Honorable Mention in the 2024 Test of Time Awards by the ACM International Conference on the Foundations of Software Engineering, a testament to the enduring impact of their research. The team was recognized at FSE 2024 at Porto de Galinhas, Brazil, on July 18.
Written by Bruce Adams
CS professor Darko Marinov and three of his former students, Qingzhou Luo, Farah Hariri, and Lamyaa Eloussi, were awarded an Honorable Mention in the 2024 Test of Time Awards by the ACM International Conference on the Foundations of Software Engineering, a testament to the enduring impact of their research. The team was recognized at FSE 2024 at Porto de Galinhas, Brazil, on July 18.
Photo Credit: FSE
Darko Marinov accepting the Test of Time Honorable Mention award at the FSE 2024 conference.
Their FSE 2014 paper "An empirical analysis of flaky tests" was cited "for conducting a rigorous empirical study analyzing non-deterministic outcomes of regression tests due to flaky tests." The paper was initially published in the Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering.
It was the first study of its kind, “looking in detail a total of 201 commits that likely fix flaky tests in 51 open-source projects.” A flaky test is a software test that yields both passing and failing results despite zero changes to the code or test. In other words, flaky tests fail to produce the same outcome with each individual test run. The non-deterministic nature of flaky tests makes debugging difficult for developers and can translate into issues for end users.
Aa Marinov's presentation at FSE 2024 states, "Flaky test failures are a big problem because they mislead developers about changes, waste developers' time, and reduce developers' trust in testing."
The Test of Time Award recognizes “highly influential papers.” As of July 2024, the ACM Digital Library lists the paper as having been cited 267 times and downloaded 1,812 times, including 273 downloads in the past year. Qingzhou Luo is now a staff software engineer at Google, Farah Hariri works as a senior software engineer at Katana Graph, and Lamyaa Eloussi is a senior software engineer at Google.