Test suite reduction for fault detection and localization: A combined approach

László Vidács, A. Beszédes, Dávid Tengeri, István Siket, T. Gyimóthy

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Citations (Scopus)

Abstract

The relation of test suites and actual faults in a software is of critical importance for timely product release. There are two particularily critical properties of test suites to this end: fault localization capability, to characterize the effort of finding the actually defective program elements, and fault detection capability which measures how probable is their manifestation and detection in the first place. While there are well established methods to predict fault detection capability (by measuring code coverage, for instance), characterization of fault localization is an emerging research topic. In this work, we investigate the effect of different test reduction methods on the performance of fault localization and detection techniques. We also provide new combined methods that incorporate both localization and detection aspects. We empirically evaluate the methods first by measuring detection and localization metrics of test suites with various reduction sizes, followed by how reduced test suites perform with actual faults. We experiment with SIR programs traditionally used in fault localization research, and extend the case study with large industrial software systems including GCC and WebKit.

Original languageEnglish
Title of host publication2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering, CSMR-WCRE 2014 - Proceedings
PublisherIEEE Computer Society
Pages204-213
Number of pages10
DOIs
Publication statusPublished - 2014
Event2014 1st Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering, CSMR-WCRE 2014 - Antwerp, Belgium
Duration: Feb 3 2014Feb 6 2014

Other

Other2014 1st Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering, CSMR-WCRE 2014
CountryBelgium
CityAntwerp
Period2/3/142/6/14

Fingerprint

Fault detection
Experiments

ASJC Scopus subject areas

  • Software

Cite this

Vidács, L., Beszédes, A., Tengeri, D., Siket, I., & Gyimóthy, T. (2014). Test suite reduction for fault detection and localization: A combined approach. In 2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering, CSMR-WCRE 2014 - Proceedings (pp. 204-213). [6747172] IEEE Computer Society. https://doi.org/10.1109/CSMR-WCRE.2014.6747172

Test suite reduction for fault detection and localization : A combined approach. / Vidács, László; Beszédes, A.; Tengeri, Dávid; Siket, István; Gyimóthy, T.

2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering, CSMR-WCRE 2014 - Proceedings. IEEE Computer Society, 2014. p. 204-213 6747172.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Vidács, L, Beszédes, A, Tengeri, D, Siket, I & Gyimóthy, T 2014, Test suite reduction for fault detection and localization: A combined approach. in 2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering, CSMR-WCRE 2014 - Proceedings., 6747172, IEEE Computer Society, pp. 204-213, 2014 1st Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering, CSMR-WCRE 2014, Antwerp, Belgium, 2/3/14. https://doi.org/10.1109/CSMR-WCRE.2014.6747172
Vidács L, Beszédes A, Tengeri D, Siket I, Gyimóthy T. Test suite reduction for fault detection and localization: A combined approach. In 2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering, CSMR-WCRE 2014 - Proceedings. IEEE Computer Society. 2014. p. 204-213. 6747172 https://doi.org/10.1109/CSMR-WCRE.2014.6747172
Vidács, László ; Beszédes, A. ; Tengeri, Dávid ; Siket, István ; Gyimóthy, T. / Test suite reduction for fault detection and localization : A combined approach. 2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering, CSMR-WCRE 2014 - Proceedings. IEEE Computer Society, 2014. pp. 204-213
@inproceedings{ba658e34c2874cf99fcf38d1281e25f7,
title = "Test suite reduction for fault detection and localization: A combined approach",
abstract = "The relation of test suites and actual faults in a software is of critical importance for timely product release. There are two particularily critical properties of test suites to this end: fault localization capability, to characterize the effort of finding the actually defective program elements, and fault detection capability which measures how probable is their manifestation and detection in the first place. While there are well established methods to predict fault detection capability (by measuring code coverage, for instance), characterization of fault localization is an emerging research topic. In this work, we investigate the effect of different test reduction methods on the performance of fault localization and detection techniques. We also provide new combined methods that incorporate both localization and detection aspects. We empirically evaluate the methods first by measuring detection and localization metrics of test suites with various reduction sizes, followed by how reduced test suites perform with actual faults. We experiment with SIR programs traditionally used in fault localization research, and extend the case study with large industrial software systems including GCC and WebKit.",
author = "L{\'a}szl{\'o} Vid{\'a}cs and A. Besz{\'e}des and D{\'a}vid Tengeri and Istv{\'a}n Siket and T. Gyim{\'o}thy",
year = "2014",
doi = "10.1109/CSMR-WCRE.2014.6747172",
language = "English",
pages = "204--213",
booktitle = "2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering, CSMR-WCRE 2014 - Proceedings",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - Test suite reduction for fault detection and localization

T2 - A combined approach

AU - Vidács, László

AU - Beszédes, A.

AU - Tengeri, Dávid

AU - Siket, István

AU - Gyimóthy, T.

PY - 2014

Y1 - 2014

N2 - The relation of test suites and actual faults in a software is of critical importance for timely product release. There are two particularily critical properties of test suites to this end: fault localization capability, to characterize the effort of finding the actually defective program elements, and fault detection capability which measures how probable is their manifestation and detection in the first place. While there are well established methods to predict fault detection capability (by measuring code coverage, for instance), characterization of fault localization is an emerging research topic. In this work, we investigate the effect of different test reduction methods on the performance of fault localization and detection techniques. We also provide new combined methods that incorporate both localization and detection aspects. We empirically evaluate the methods first by measuring detection and localization metrics of test suites with various reduction sizes, followed by how reduced test suites perform with actual faults. We experiment with SIR programs traditionally used in fault localization research, and extend the case study with large industrial software systems including GCC and WebKit.

AB - The relation of test suites and actual faults in a software is of critical importance for timely product release. There are two particularily critical properties of test suites to this end: fault localization capability, to characterize the effort of finding the actually defective program elements, and fault detection capability which measures how probable is their manifestation and detection in the first place. While there are well established methods to predict fault detection capability (by measuring code coverage, for instance), characterization of fault localization is an emerging research topic. In this work, we investigate the effect of different test reduction methods on the performance of fault localization and detection techniques. We also provide new combined methods that incorporate both localization and detection aspects. We empirically evaluate the methods first by measuring detection and localization metrics of test suites with various reduction sizes, followed by how reduced test suites perform with actual faults. We experiment with SIR programs traditionally used in fault localization research, and extend the case study with large industrial software systems including GCC and WebKit.

UR - http://www.scopus.com/inward/record.url?scp=84898483837&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84898483837&partnerID=8YFLogxK

U2 - 10.1109/CSMR-WCRE.2014.6747172

DO - 10.1109/CSMR-WCRE.2014.6747172

M3 - Conference contribution

AN - SCOPUS:84898483837

SP - 204

EP - 213

BT - 2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering, CSMR-WCRE 2014 - Proceedings

PB - IEEE Computer Society

ER -