SCAS-Fuzzy: uma estratégia semiautomática para seleção de estudos primários em estudos secundários

Abstract

Context: Systematic review and systematic mapping are secondary studies used to identify and aggregate relevant literature evidence on a research question of interest. One of the activities associated with secondary studies is the selection of primary studies, which is a manual activity and may require great effort from the researchers. The quality of the selection of primary studies directly affects the overall quality of the secondary studies. Objective: To propose a strategy called SCAS-Fuzzy (Score Citation semi-Automatic Selection using Fuzzy set) to automate part of the activity of selection of primary studies, minimizing the effort required in this activity, but maintaining the quality of the selection. Methodology: it was proposed a semi-automatic strategy for the selection of primary studies based on two functionalities: the score and whether a study is cited or not, which was called SCAS. It was evaluated through a case study and an experiment, which showed promising results. Then, ways to improve it were investigated and a citation coefficient was created, which now considers the number of citations caught by a study and the year of its publication, besides using fuzzy logic for classification of studies, which is now based on their scores and citation coefficients. The improved strategy, called SCAS-Fuzzy, was evaluated through a case study. Results: the case study showed that, for the five systematic reviews considered, the general effort reduction applying the SCAS-Fuzzy strategy was 39.1% and the error percentage was 0.3% for automatically excluding studies and 3.3% for automatically including studies when compared to manual review, showing a substantial level of agreement with reviewers. Conclusion: based on the results it is possible to conclude that the SCAS-Fuzzy strategy provided satisfactory results to reduce the effort of the initial selection activity and with a very low amount of evidence loss, maintaining the quality of the secondary study, also presenting better results in general in relation to the original defined SCAS strategy.