Code coverage-based regression test selection and prioritization in WebKit

A. Beszédes, Tamas Gergely, Lajos Schrettner, Judit Jasz, Laszlo Lango, T. Gyimóthy

Research output: Chapter in Book/Report/Conference proceedingConference contribution

16 Citations (Scopus)

Abstract

Automated regression testing is often crucial in order to maintain the quality of a continuously evolving software system. However, in many cases regression test suites tend to grow too large to be suitable for full re-execution at each change of the software. In this case selective retesting can be applied to reduce the testing cost while maintaining similar defect detection capability. One of the basic test selection methods is the one based on code coverage information, where only those tests are included that cover some parts of the changes. We experimentally applied this method to the open source web browser engine project WebKit to find out the technical difficulties and the expected benefits if this method is to be introduced into the actual build process. Although the principle is simple, we had to solve a number of technical issues, so we report how this method was adapted to be used in the official build environment. Second, we present results about the selection capabilities for a selected set of revisions of WebKit, which are promising. We also applied different test case prioritization strategies to further reduce the number of tests to execute. We explain these strategies and compare their usefulness in terms of defect detection and test suite reduction.

Original languageEnglish
Title of host publicationIEEE International Conference on Software Maintenance, ICSM
Pages46-55
Number of pages10
DOIs
Publication statusPublished - 2012
Event28th IEEE International Conference on Software Maintenance, ICSM 2012 - Riva del Garda,Trento, Italy
Duration: Sep 23 2012Sep 28 2012

Other

Other28th IEEE International Conference on Software Maintenance, ICSM 2012
CountryItaly
CityRiva del Garda,Trento
Period9/23/129/28/12

Fingerprint

Web browsers
Testing
Engines
Costs
Defect detection

Keywords

  • code coverage
  • Regression testing
  • test case selection
  • test prioritization
  • test quality
  • WebKit

ASJC Scopus subject areas

  • Software

Cite this

Beszédes, A., Gergely, T., Schrettner, L., Jasz, J., Lango, L., & Gyimóthy, T. (2012). Code coverage-based regression test selection and prioritization in WebKit. In IEEE International Conference on Software Maintenance, ICSM (pp. 46-55). [6405252] https://doi.org/10.1109/ICSM.2012.6405252

Code coverage-based regression test selection and prioritization in WebKit. / Beszédes, A.; Gergely, Tamas; Schrettner, Lajos; Jasz, Judit; Lango, Laszlo; Gyimóthy, T.

IEEE International Conference on Software Maintenance, ICSM. 2012. p. 46-55 6405252.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Beszédes, A, Gergely, T, Schrettner, L, Jasz, J, Lango, L & Gyimóthy, T 2012, Code coverage-based regression test selection and prioritization in WebKit. in IEEE International Conference on Software Maintenance, ICSM., 6405252, pp. 46-55, 28th IEEE International Conference on Software Maintenance, ICSM 2012, Riva del Garda,Trento, Italy, 9/23/12. https://doi.org/10.1109/ICSM.2012.6405252
Beszédes A, Gergely T, Schrettner L, Jasz J, Lango L, Gyimóthy T. Code coverage-based regression test selection and prioritization in WebKit. In IEEE International Conference on Software Maintenance, ICSM. 2012. p. 46-55. 6405252 https://doi.org/10.1109/ICSM.2012.6405252
Beszédes, A. ; Gergely, Tamas ; Schrettner, Lajos ; Jasz, Judit ; Lango, Laszlo ; Gyimóthy, T. / Code coverage-based regression test selection and prioritization in WebKit. IEEE International Conference on Software Maintenance, ICSM. 2012. pp. 46-55
@inproceedings{6fe36d472bef4e459249a983b5301f16,
title = "Code coverage-based regression test selection and prioritization in WebKit",
abstract = "Automated regression testing is often crucial in order to maintain the quality of a continuously evolving software system. However, in many cases regression test suites tend to grow too large to be suitable for full re-execution at each change of the software. In this case selective retesting can be applied to reduce the testing cost while maintaining similar defect detection capability. One of the basic test selection methods is the one based on code coverage information, where only those tests are included that cover some parts of the changes. We experimentally applied this method to the open source web browser engine project WebKit to find out the technical difficulties and the expected benefits if this method is to be introduced into the actual build process. Although the principle is simple, we had to solve a number of technical issues, so we report how this method was adapted to be used in the official build environment. Second, we present results about the selection capabilities for a selected set of revisions of WebKit, which are promising. We also applied different test case prioritization strategies to further reduce the number of tests to execute. We explain these strategies and compare their usefulness in terms of defect detection and test suite reduction.",
keywords = "code coverage, Regression testing, test case selection, test prioritization, test quality, WebKit",
author = "A. Besz{\'e}des and Tamas Gergely and Lajos Schrettner and Judit Jasz and Laszlo Lango and T. Gyim{\'o}thy",
year = "2012",
doi = "10.1109/ICSM.2012.6405252",
language = "English",
isbn = "9781467323123",
pages = "46--55",
booktitle = "IEEE International Conference on Software Maintenance, ICSM",

}

TY - GEN

T1 - Code coverage-based regression test selection and prioritization in WebKit

AU - Beszédes, A.

AU - Gergely, Tamas

AU - Schrettner, Lajos

AU - Jasz, Judit

AU - Lango, Laszlo

AU - Gyimóthy, T.

PY - 2012

Y1 - 2012

N2 - Automated regression testing is often crucial in order to maintain the quality of a continuously evolving software system. However, in many cases regression test suites tend to grow too large to be suitable for full re-execution at each change of the software. In this case selective retesting can be applied to reduce the testing cost while maintaining similar defect detection capability. One of the basic test selection methods is the one based on code coverage information, where only those tests are included that cover some parts of the changes. We experimentally applied this method to the open source web browser engine project WebKit to find out the technical difficulties and the expected benefits if this method is to be introduced into the actual build process. Although the principle is simple, we had to solve a number of technical issues, so we report how this method was adapted to be used in the official build environment. Second, we present results about the selection capabilities for a selected set of revisions of WebKit, which are promising. We also applied different test case prioritization strategies to further reduce the number of tests to execute. We explain these strategies and compare their usefulness in terms of defect detection and test suite reduction.

AB - Automated regression testing is often crucial in order to maintain the quality of a continuously evolving software system. However, in many cases regression test suites tend to grow too large to be suitable for full re-execution at each change of the software. In this case selective retesting can be applied to reduce the testing cost while maintaining similar defect detection capability. One of the basic test selection methods is the one based on code coverage information, where only those tests are included that cover some parts of the changes. We experimentally applied this method to the open source web browser engine project WebKit to find out the technical difficulties and the expected benefits if this method is to be introduced into the actual build process. Although the principle is simple, we had to solve a number of technical issues, so we report how this method was adapted to be used in the official build environment. Second, we present results about the selection capabilities for a selected set of revisions of WebKit, which are promising. We also applied different test case prioritization strategies to further reduce the number of tests to execute. We explain these strategies and compare their usefulness in terms of defect detection and test suite reduction.

KW - code coverage

KW - Regression testing

KW - test case selection

KW - test prioritization

KW - test quality

KW - WebKit

UR - http://www.scopus.com/inward/record.url?scp=84873124514&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84873124514&partnerID=8YFLogxK

U2 - 10.1109/ICSM.2012.6405252

DO - 10.1109/ICSM.2012.6405252

M3 - Conference contribution

SN - 9781467323123

SP - 46

EP - 55

BT - IEEE International Conference on Software Maintenance, ICSM

ER -