Investigação de estratégia para redução de custo do teste de mutação com apoio de similaridade entre programas

Abstract

Context: The mutation test criterion stands out as an important alternative for evaluating both the software to be tested and the test set to be used. This criterion is considered effective, but it has some gaps that must be filled in relation to its efficiency, due to its high execution cost. There is a relatively high set of techniques that reduce their execution costs, aiming at their use in industry, but their results are not generally generalizable. Some studies related to cost reduction techniques presented heuristics whose objective is to calculate the similarities of the programs used, but not deeply studied. Objective: This work presents a systematic review of the literature that seeks to present the state of the art in mutation testing, the definition, creation and population of a database of artifacts, the implementation of an automated process for defining the Essential Set of mutation operators to help reduce the cost of running mutation testing. Methodology: The implementation presented in this work deals with programs written in the Java language and includes a set of variants that relate to three types of program abstractions (original source code, obfuscated source code, and internal complexity metrics), three calculation strategies of similarity (clustering, distance functions between strings, and plagiarism) and a mutation cost reduction approach (inspired by best operators and essential set techniques). The evaluation presented, using 221 classes written in the Java language, covers 20 configurations varying the techniques of program abstractions and similarities. Results: The implementation of a knowledge base of structured artifacts, automation of the technique of defining the Essential Set of Operators, a cross-comparison involving the clusters formed that tend to achieve high effectiveness in predicting the best mutation operators and essential set for untested programs with mutation testing and a statistical analysis evaluating the similarity between configuration results. Conclusions: Considering the configurations selected for the experiments, the results presented for the combinations using the essential sets technique for the comparison of effectiveness, highlighting the configurations that presented the best set of operators, use of new sizes for the reference clusters, an analytical analysis of the results associated with a statistical analysis using the variance technique to verify the equivalence between the sets of results.