Beyond code coverage - An approach for test suite assessment and improvement

Dávid Tengeri, A. Beszédes, Tamás Gergely, László Vidács, David Hávas, T. Gyimóthy

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)

Abstract

Code coverage is successfully used to guide white box test design and evaluate the respective test completeness. However, simple overall coverage ratios are often not precise enough to effectively help when a (regression) test suite needs to be reassessed and evolved after software change. We present an approach for test suite assessment and improvement that utilizes code coverage information, but on a more detailed level and adds further evaluation aspects derived from the coverage. The main use of the method is to aid various test suite evolution situations such as removal, refactoring and extension of test cases as a result of code change or test suite efficiency enhancement. We define various metrics to express different properties of test suites beyond simple code coverage ratios, and present the assessment and improvement process as an iterative application of different improvement goals and more specific sub-activities. The method is demonstrated by applying it to improve the tests of one of our experimental systems.

Original languageEnglish
Title of host publication2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Print)9781479918850
DOIs
Publication statusPublished - May 13 2015
Event2015 8th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Graz, Austria
Duration: Apr 13 2015Apr 17 2015

Other

Other2015 8th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015
CountryAustria
CityGraz
Period4/13/154/17/15

Keywords

  • code coverage
  • regression testing
  • test suite evolution
  • test suite quality
  • test suite refactoring
  • white box testing metrics

ASJC Scopus subject areas

  • Software

Cite this

Tengeri, D., Beszédes, A., Gergely, T., Vidács, L., Hávas, D., & Gyimóthy, T. (2015). Beyond code coverage - An approach for test suite assessment and improvement. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings [7107476] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICSTW.2015.7107476

Beyond code coverage - An approach for test suite assessment and improvement. / Tengeri, Dávid; Beszédes, A.; Gergely, Tamás; Vidács, László; Hávas, David; Gyimóthy, T.

2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2015. 7107476.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Tengeri, D, Beszédes, A, Gergely, T, Vidács, L, Hávas, D & Gyimóthy, T 2015, Beyond code coverage - An approach for test suite assessment and improvement. in 2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings., 7107476, Institute of Electrical and Electronics Engineers Inc., 2015 8th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015, Graz, Austria, 4/13/15. https://doi.org/10.1109/ICSTW.2015.7107476
Tengeri D, Beszédes A, Gergely T, Vidács L, Hávas D, Gyimóthy T. Beyond code coverage - An approach for test suite assessment and improvement. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2015. 7107476 https://doi.org/10.1109/ICSTW.2015.7107476
Tengeri, Dávid ; Beszédes, A. ; Gergely, Tamás ; Vidács, László ; Hávas, David ; Gyimóthy, T. / Beyond code coverage - An approach for test suite assessment and improvement. 2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2015.
@inproceedings{4b886c240d934584bde08a1585968c77,
title = "Beyond code coverage - An approach for test suite assessment and improvement",
abstract = "Code coverage is successfully used to guide white box test design and evaluate the respective test completeness. However, simple overall coverage ratios are often not precise enough to effectively help when a (regression) test suite needs to be reassessed and evolved after software change. We present an approach for test suite assessment and improvement that utilizes code coverage information, but on a more detailed level and adds further evaluation aspects derived from the coverage. The main use of the method is to aid various test suite evolution situations such as removal, refactoring and extension of test cases as a result of code change or test suite efficiency enhancement. We define various metrics to express different properties of test suites beyond simple code coverage ratios, and present the assessment and improvement process as an iterative application of different improvement goals and more specific sub-activities. The method is demonstrated by applying it to improve the tests of one of our experimental systems.",
keywords = "code coverage, regression testing, test suite evolution, test suite quality, test suite refactoring, white box testing metrics",
author = "D{\'a}vid Tengeri and A. Besz{\'e}des and Tam{\'a}s Gergely and L{\'a}szl{\'o} Vid{\'a}cs and David H{\'a}vas and T. Gyim{\'o}thy",
year = "2015",
month = "5",
day = "13",
doi = "10.1109/ICSTW.2015.7107476",
language = "English",
isbn = "9781479918850",
booktitle = "2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Beyond code coverage - An approach for test suite assessment and improvement

AU - Tengeri, Dávid

AU - Beszédes, A.

AU - Gergely, Tamás

AU - Vidács, László

AU - Hávas, David

AU - Gyimóthy, T.

PY - 2015/5/13

Y1 - 2015/5/13

N2 - Code coverage is successfully used to guide white box test design and evaluate the respective test completeness. However, simple overall coverage ratios are often not precise enough to effectively help when a (regression) test suite needs to be reassessed and evolved after software change. We present an approach for test suite assessment and improvement that utilizes code coverage information, but on a more detailed level and adds further evaluation aspects derived from the coverage. The main use of the method is to aid various test suite evolution situations such as removal, refactoring and extension of test cases as a result of code change or test suite efficiency enhancement. We define various metrics to express different properties of test suites beyond simple code coverage ratios, and present the assessment and improvement process as an iterative application of different improvement goals and more specific sub-activities. The method is demonstrated by applying it to improve the tests of one of our experimental systems.

AB - Code coverage is successfully used to guide white box test design and evaluate the respective test completeness. However, simple overall coverage ratios are often not precise enough to effectively help when a (regression) test suite needs to be reassessed and evolved after software change. We present an approach for test suite assessment and improvement that utilizes code coverage information, but on a more detailed level and adds further evaluation aspects derived from the coverage. The main use of the method is to aid various test suite evolution situations such as removal, refactoring and extension of test cases as a result of code change or test suite efficiency enhancement. We define various metrics to express different properties of test suites beyond simple code coverage ratios, and present the assessment and improvement process as an iterative application of different improvement goals and more specific sub-activities. The method is demonstrated by applying it to improve the tests of one of our experimental systems.

KW - code coverage

KW - regression testing

KW - test suite evolution

KW - test suite quality

KW - test suite refactoring

KW - white box testing metrics

UR - http://www.scopus.com/inward/record.url?scp=84934301413&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84934301413&partnerID=8YFLogxK

U2 - 10.1109/ICSTW.2015.7107476

DO - 10.1109/ICSTW.2015.7107476

M3 - Conference contribution

AN - SCOPUS:84934301413

SN - 9781479918850

BT - 2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -