Geração e avaliação de dados sintéticos para aplicações em saúde mental

Abstract

Annotated corpora in the mental health domain are scarce and frequently cannot be publicly redistributed due to ethical and legal restrictions associated with the sensitivity of the data involved, which limits the reproducibility of experiments and collaborative progress in the field. This work proposes and evaluates a synthetic data generation pipeline for the Amive corpus, composed of anonymous social media posts annotated with depression symptoms, aiming to enable the public distribution of a functional corpus without exposing the original texts of the users who produced them. The pipeline combines machine translation via a language model (Portuguese to English and back-translation to Portuguese), paraphrase generation with the PEGASUS model through diverse beam search, and selection of the best candidate paraphrase based on semantic similarity computed by BERTScore, using BERTimbau as the representation model. The evaluation of the generated synthetic corpus, focused on the Sadness/Depressed Mood symptom, is conducted along two complementary lines: an extrinsic evaluation, in which binary classifiers of different natures (Logistic Regression, SVM, Naive Bayes, and Random Forest) are trained and tested across four scenarios combining original and synthetic data; and an intrinsic evaluation, based on lexical diversity metrics and the part-of-speech distribution of the generated corpus, complemented by a qualitative analysis of representative examples. Results indicate that the synthetic corpus largely preserves the utility of the original corpus for the classification task, with particular emphasis on recall, a metric considered more critical in mental health screening applications, where failing to identify a positive case tends to be more costly than a false alarm. However, a more consistent drop in precision is observed for classifiers trained exclusively on synthetic data when evaluated on real data, along with evidence that part of the lexical diversity introduced stems from a tendency of the pipeline to prune content in syntactically complex sentences, rather than from genuine reformulation alone. The qualitative analysis further reveals recurring patterns of discursive nuance loss, such as the depersonalization of autobiographical accounts and the literalization of metaphorical constructions. It is concluded that the proposed pipeline constitutes a viable, though not limitation-free, mechanism for generating synthetic corpora in Portuguese within the mental health domain, offering a concrete alternative to the unavailability of annotated data in this field.