Skip to main navigation Skip to search Skip to main content

Multi-objective test case prioritization in highly configurable systems: A case study

  • University of Seville
  • Johannes Kepler University Linz

Research output: Contribution to journal › Journal Article › peer-review

88 Citations (Scopus)

Abstract

Test case prioritization schedules test cases for execution in an order that attempts to accelerate the detection of faults. The order of test cases is determined by prioritization objectives such as covering code or critical components as rapidly as possible. The importance of this technique has been recognized in the context of Highly-Configurable Systems (HCSs), where the potentially huge number of configurations makes testing extremely challenging. However, current approaches for test case prioritization in HCSs suffer from two main limitations. First, the prioritization is usually driven by a single objective which neglects the potential benefits of combining multiple criteria to guide the detection of faults. Second, instead of using industry-strength case studies, evaluations are conducted using synthetic data, which provides no information about the effectiveness of different prioritization objectives. In this paper, we address both limitations by studying 63 combinations of up to three prioritization objectives in accelerating the detection of faults in the Drupal framework. Results show that non–functional properties such as the number of changes in the features are more effective than functional metrics extracted from the configuration model. Results also suggest that multi-objective prioritization typically results in faster fault detection than mono-objective prioritization.

Original languageEnglish
Pages (from-to)287-310
Number of pages24
JournalJournal of Systems and Software
Volume122
DOIs
Publication statusPublished - 1 Dec 2016
Externally publishedYes

Keywords

  • Automated software testing
  • Highly-configurable systems
  • Test case prioritization
  • Variability

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'Multi-objective test case prioritization in highly configurable systems: A case study'. These topics are generated from the title and abstract of the publication. Together, they form a unique fingerprint.

Cite this