Test suite evaluation using code coverage based metrics

F. Horváth, Béla Vancsics, László Vidács, A. Beszédes, Dávid Tengeri, Tamás Gergely, T. Gyimóthy

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

Regression test suites of evolving software systems are often crucial to maintaining software quality in the long term. They have to be effective in terms of detecting faults and helping their localization. However, to gain knowledge of such capabilities of test suites is usually difficult. We propose a method for deeper understanding of a test suite and its relation to the program code it is intended to test. The basic idea is to decompose the test suite and the program code into coherent logical groups which are easier to analyze and understand. Coverage and partition metrics are then extracted directly from code coverage information to characterize a test suite and its constituents. We also use heat-map tables for test suite assessment both at the system level and at the level of logical groups. We employ these metrics to analyze and evaluate the regression test suite of the WebKit system, an industrial size browser engine with an extensive set of 27,000 tests.

Original languageEnglish
Title of host publicationCEUR Workshop Proceedings
PublisherCEUR-WS
Pages46-60
Number of pages15
Volume1525
Publication statusPublished - 2015
Event14th Symposium on Programming Languages and Software Tools, SPLST 2015 - Tampere, Finland
Duration: Oct 9 2015Oct 10 2015

Other

Other14th Symposium on Programming Languages and Software Tools, SPLST 2015
CountryFinland
CityTampere
Period10/9/1510/10/15

Fingerprint

Engines
Hot Temperature

Keywords

  • Code coverage
  • Regression testing
  • Test metrics
  • Test suite evaluation

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

Horváth, F., Vancsics, B., Vidács, L., Beszédes, A., Tengeri, D., Gergely, T., & Gyimóthy, T. (2015). Test suite evaluation using code coverage based metrics. In CEUR Workshop Proceedings (Vol. 1525, pp. 46-60). CEUR-WS.

Test suite evaluation using code coverage based metrics. / Horváth, F.; Vancsics, Béla; Vidács, László; Beszédes, A.; Tengeri, Dávid; Gergely, Tamás; Gyimóthy, T.

CEUR Workshop Proceedings. Vol. 1525 CEUR-WS, 2015. p. 46-60.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Horváth, F, Vancsics, B, Vidács, L, Beszédes, A, Tengeri, D, Gergely, T & Gyimóthy, T 2015, Test suite evaluation using code coverage based metrics. in CEUR Workshop Proceedings. vol. 1525, CEUR-WS, pp. 46-60, 14th Symposium on Programming Languages and Software Tools, SPLST 2015, Tampere, Finland, 10/9/15.
Horváth F, Vancsics B, Vidács L, Beszédes A, Tengeri D, Gergely T et al. Test suite evaluation using code coverage based metrics. In CEUR Workshop Proceedings. Vol. 1525. CEUR-WS. 2015. p. 46-60
Horváth, F. ; Vancsics, Béla ; Vidács, László ; Beszédes, A. ; Tengeri, Dávid ; Gergely, Tamás ; Gyimóthy, T. / Test suite evaluation using code coverage based metrics. CEUR Workshop Proceedings. Vol. 1525 CEUR-WS, 2015. pp. 46-60
@inproceedings{e9fd4298871b4bffb9c624699e9c273a,
title = "Test suite evaluation using code coverage based metrics",
abstract = "Regression test suites of evolving software systems are often crucial to maintaining software quality in the long term. They have to be effective in terms of detecting faults and helping their localization. However, to gain knowledge of such capabilities of test suites is usually difficult. We propose a method for deeper understanding of a test suite and its relation to the program code it is intended to test. The basic idea is to decompose the test suite and the program code into coherent logical groups which are easier to analyze and understand. Coverage and partition metrics are then extracted directly from code coverage information to characterize a test suite and its constituents. We also use heat-map tables for test suite assessment both at the system level and at the level of logical groups. We employ these metrics to analyze and evaluate the regression test suite of the WebKit system, an industrial size browser engine with an extensive set of 27,000 tests.",
keywords = "Code coverage, Regression testing, Test metrics, Test suite evaluation",
author = "F. Horv{\'a}th and B{\'e}la Vancsics and L{\'a}szl{\'o} Vid{\'a}cs and A. Besz{\'e}des and D{\'a}vid Tengeri and Tam{\'a}s Gergely and T. Gyim{\'o}thy",
year = "2015",
language = "English",
volume = "1525",
pages = "46--60",
booktitle = "CEUR Workshop Proceedings",
publisher = "CEUR-WS",

}

TY - GEN

T1 - Test suite evaluation using code coverage based metrics

AU - Horváth, F.

AU - Vancsics, Béla

AU - Vidács, László

AU - Beszédes, A.

AU - Tengeri, Dávid

AU - Gergely, Tamás

AU - Gyimóthy, T.

PY - 2015

Y1 - 2015

N2 - Regression test suites of evolving software systems are often crucial to maintaining software quality in the long term. They have to be effective in terms of detecting faults and helping their localization. However, to gain knowledge of such capabilities of test suites is usually difficult. We propose a method for deeper understanding of a test suite and its relation to the program code it is intended to test. The basic idea is to decompose the test suite and the program code into coherent logical groups which are easier to analyze and understand. Coverage and partition metrics are then extracted directly from code coverage information to characterize a test suite and its constituents. We also use heat-map tables for test suite assessment both at the system level and at the level of logical groups. We employ these metrics to analyze and evaluate the regression test suite of the WebKit system, an industrial size browser engine with an extensive set of 27,000 tests.

AB - Regression test suites of evolving software systems are often crucial to maintaining software quality in the long term. They have to be effective in terms of detecting faults and helping their localization. However, to gain knowledge of such capabilities of test suites is usually difficult. We propose a method for deeper understanding of a test suite and its relation to the program code it is intended to test. The basic idea is to decompose the test suite and the program code into coherent logical groups which are easier to analyze and understand. Coverage and partition metrics are then extracted directly from code coverage information to characterize a test suite and its constituents. We also use heat-map tables for test suite assessment both at the system level and at the level of logical groups. We employ these metrics to analyze and evaluate the regression test suite of the WebKit system, an industrial size browser engine with an extensive set of 27,000 tests.

KW - Code coverage

KW - Regression testing

KW - Test metrics

KW - Test suite evaluation

UR - http://www.scopus.com/inward/record.url?scp=84962535014&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84962535014&partnerID=8YFLogxK

M3 - Conference contribution

VL - 1525

SP - 46

EP - 60

BT - CEUR Workshop Proceedings

PB - CEUR-WS

ER -