title   
  

Feedback driven adaptive combinatorial testing

Dumlu, Emine and Yılmaz, Cemal and Cohen, Myra B. and Porter, Adam (2011) Feedback driven adaptive combinatorial testing. In: ACM International Symposium on Software Testing and Analysis (ISSTA 2011), Toronto, ON, Canada

[img]PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
332Kb

Abstract

The configuration spaces of modern software systems are too large to test exhaustively. Combinatorial interaction testing (CIT) approaches, such as covering arrays, systematically sample the configuration space and test only the selected configurations. The basic justification for CIT approaches is that they can cost-effectively exercise all system behaviors caused by the settings of t or fewer options. We conjecture, however, that in practice many such behaviors are not actually tested because of masking effects – failures that perturb execution so as to prevent some behaviors from being exercised. In this work we present a feedback-driven, adaptive, combinatorial testing approach aimed at detecting and working around masking effects. At each iteration we detect potential masking effects, heuristically isolate their likely causes, and then generate new covering arrays that allow previously masked combinations to be tested in the subsequent iteration. We empirically assess the effectiveness of the proposed approach on two large widely used open source software systems. Our results suggest that masking effects do exist and that our approach provides a promising and efficient way to work around them.

Item Type:Papers in Conference Proceedings
Uncontrolled Keywords:Combinatorial testing, adaptive testing, covering arrays, software quality assurance
Subjects:Q Science > Q Science (General)
ID Code:16908
Deposited By:Cemal Yılmaz
Deposited On:20 Sep 2011 16:13
Last Modified:20 Sep 2011 16:13

Repository Staff Only: item control page