Avaliação de bandits contextuais para recomendação: temporalidade e limitações

Abstract

In a digital environment where users are daily exposed to a massive volume of content, recommender systems play an essential role in filtering and personalizing information. These systems, however, face the classic dilemma between exploration (introducing new items) and exploitation (reinforcing known preferences). Finding the ideal balance between these two behaviors remains one of the major challenges in the field, especially in adaptive approaches such as contextual Multi-Armed Bandits (MAB), which learn continuously from user interactions over time. This work began with the investigation of different linear MAB algorithms in recommendation and offline evaluation scenarios. During these experiments, a systematic bias was observed in traditional metrics, favoring purely greedy methods (without exploration) and compromising both the analysis of exploratory strategies and the fair comparison between policies. To overcome these limitations, a new online evaluation methodology was proposed and implemented in a simulated environment. The KuaiSim simulator, based on the KuaiRand dataset, was extensively adapted to support multi-session interactions, contextual modeling, and temporal dependency. This infrastructure enabled a more realistic investigation of how temporal factors influence recommendation behavior. Building upon this new environment, the temporal method Time-Aware LinBoltzmann was developed. It combines linear models with Boltzmann exploration and dynamically adjusts the temperature parameter according to the time interval between user interactions. The underlying intuition is that users who return quickly tend to prefer recommendations aligned with their previous interests, while longer return intervals may indicate a greater willingness to explore new options. Experiments conducted in the simulator show that incorporating temporal information improves diversity and coverage metrics compared to baseline methods, suggesting that time is a relevant signal for modulating the balance between exploration and exploitation. These findings open new perspectives for recommender systems that are sensitive to users’ temporal behavior. The main contributions of this work include: (i) the identification and analysis of bias in offline evaluation protocols for linear MABs; (ii) the development of an online simulation framework based on temporal sessions; (iii) the proposal of the Time-Aware LinBoltzmann algorithm, which introduces temporal awareness into the exploration process; and (iv) the discussion of implications, limitations, and future opportunities for incorporating temporal information into recommender systems.