SiMut: um framework automatizado para apoiar a redução do custo do teste de mutação com base em similaridade de programas

Abstract

Context: Software testing is essential to ensure that the program actually correctly meets the user’s needs. Aware of the importance of this step, researchers have proposed several ways to identify program faults. Among the fault identification techniques, there is the mutation analysis criterion, or mutation testing, which has proven to be effective for fault identification, but, as it demands a high application cost, it is little efficient and hence avoided in the software industry. Given the potential of this criterion, researchers sought and presented ways to reduce the cost of mutation testing to make it more viable. However, as verified by several researchers, the results produced by cost reduction techniques for mutation testing applied in a program are valid only for the programs that were targeted in the performed experiments. Recently, some researchers, in their studies, have relied on the use of similarity between programs as a way to reuse experiences acquired in programs already tested with mutation testing. However, this is an approach that still lacks experiments using various forms of similarity. Objective: This work presents an implementation and evaluation of a framework called SiMut. This framework was introduced in a previous study with the aim of helping to reduce the cost of testing a program based on a group of similar programs previously tested with mutation. Methodology: The implementation presented in this work deals with programs written in the Java language and includes a set of variants that relate to three types of program abstractions (original source code, obfuscated source code, and internal complexity metrics), three similarity calculation strategies (clustering, distance functions between strings, and plagiarism) and a mutation cost reduction approach (inspired by Selective Mutation). The presented evaluation, using 35 small programs written in the Java language, covers 20 configurations varying the techniques of abstractions and similarities. Results: A cross-comparison involving the formed clusters and a comparison with randomly formed clusters points to configurations that tend to achieve high effectiveness in predicting the best mutation operators for programs not tested with mutation. Conclusions: Considering the configurations selected for the experiments, the results presented several combinations that are effective to predict the best mutation operators for untested programs, where the combinations that involved distance functions between strings stood out. Regarding the program abstractions used to calculate similarity, the original source code seems to be as relevant as the obfuscated source code and internal complexity metrics.