Detecção de mudança de distribuição em dados sequenciais - Andrade, Douglas Decicino de (Universidade Federal de São Carlos, 2025)
This work aims to study the performance of the conformal method for distribution change detection (dataset shift) in sequential data, where observations are obtained sequentially over time. In particular, we consider cases where the observations are univariate, multivariate, and provided in batches (datasets) in predictive contexts. The conformal method generates p-values at each time step, which provide evidence regarding a potential change in the data distribution. Unlike classical approaches, such as the Cumulative Sum Control Chart (CUSUM), Shirayev-Roberts, and Posterior Probability methods, the conformal method does not rely on parametric assumptions about the data distribution before and after the change occurs. The conformal method was applied using different nonconformity measures proposed in this work, including those based on Kullback-Leibler divergence and various distances in Euclidean spaces constructed through statistical depth measures. The effectiveness of this approach for detecting distributional changes was evaluated in different empirical studies by measuring the average detection delay and the proportion of false alarms. It is expected that this research can contribute to the application of this method in the online monitoring of machine learning models in dynamic environments, allowing the identification of the appropriate time to recalibrate or retrain algorithms in a more reliable manner.
Distribution shift detection in sequential data
Keywords: Dataset shift; Detecção de mudanças de distribuição; Detecção de ponto de mudança; Dados sequenciais; Processamento em lote; Aprendizagem de máquina; Dataset Distribution change detection; Change point detection; Sequential data; Batch processing; Machine learning; CIENCIAS EXATAS E DA TERRA::PROBABILIDADE E ESTATISTICA::ESTATISTICA::ANALISE DE DADOS; CIENCIAS EXATAS E DA TERRA::PROBABILIDADE E ESTATISTICA; CIENCIAS EXATAS E DA TERRA::PROBABILIDADE E ESTATISTICA::ESTATISTICA::ANALISE MULTIVARIADA; CIENCIAS EXATAS E DA TERRA::PROBABILIDADE E ESTATISTICA::ESTATISTICA::FUNDAMENTOS DA ESTATISTICA; CIENCIAS EXATAS E DA TERRA::PROBABILIDADE E ESTATISTICA::ESTATISTICA::INFERENCIA EM PROCESSOS ESTOCASTICOS; CIENCIAS EXATAS E DA TERRA::PROBABILIDADE E ESTATISTICA::ESTATISTICA::INFERENCIA NAO-PARAMETRICA; CIENCIAS EXATAS E DA TERRA::PROBABILIDADE E ESTATISTICA::ESTATISTICA::INFERENCIA PARAMETRICA