Técnicas de quantificação de incerteza em redes neurais para densidades condicionais por meio de modelos de misturas

Abstract

The uncertainty in statistical learning models is divided into two main types: aleatoric uncertainty, related to the intrinsic variability of the data, and epistemic uncertainty, caused by incomplete knowledge about the data-generating process, often associated with data scarcity. Conditional density estimation allows for good quantification of aleatoric uncertainty. However, quantifying epistemic uncertainty traditionally requires Bayesian methods, which, although theoretically well-founded, entail high computational costs in high dimensions. Our interest in this problem arose from the scarcity of works addressing epistemic uncertainty and the search for computationally efficient and scalable alternatives. Our contribution focuses on using neural networks with dropout in mixture models, demonstrating that this approach: approximates Bayesian inference via Monte Carlo dropout, incorporating epistemic uncertainty; significantly reduces computational costs compared to Bayesian neural networks; and naturally integrates with Conformal Prediction regions to produce predictive intervals with theoretical coverage guarantees.