Dissimilar test cases have been proven to be effective to reveal faults in software systems. In the Software Product Line (SPL) context, this criteria has been applied successfully to mimic combinatorial interaction testing in an efficient and scalable manner by selecting and prioritising most dissimilar configurations of feature models using evolutionary algorithms. In this paper, we extend dissimilarity to behavioural SPL models (FTS) in a search-based approach, and evaluate its effectiveness in terms of product and fault coverage. We investigate different distances as well as as single-objective algorithms, (dissimilarity on actions, random, all-actions). Our results on four case studies show the relevance of dissimilarity-based test generation for behavioural SPL models, especially on the largest case-study where no other approach can match it.
The different models and tools used for the experiment may be downloaded here:
The complete results per model may be downloaded here.
Results may be download here. Each line corresponds to the execution of one test set. The meaning of the different fields is described hereafter: