Multimodal classification for detecting products that do not comply with the Americanas S.A.'s marketplace sales policies

Abstract

Multimodal learning for the e-commerce domain, some classification methods are ne- eded for categorization, information retrieval and product recommendations, which are generally composed of different modalities: images and texts. Due to large diversification in the characteristics of these modalities or the absence/incompleteness of information (for example, incomplete product attributes), classification methods face many difficulties in dealing with this information in order to improve their classification. Thus, this work was carried out to investigate the multimodal learning in visual and textual modalities for e-commerce. Our experiments show good results for classification of products from “Adult” and “Illegal Devices” categories, which is part of the dataset provided by the partner company of this project. In these experiments, training was carried out for the specific modalities, deriving text and image models, as well as the fusion of the two mo- dalities in a multimodal model. The best models were the binary textual models trained taking into account product titles and descriptions: TD bin-adult (with a recall of 98%) and TD bin-illegal (with a recall of 95 %). We have some insights about the multimo- dal classification, mainly for the visual modality which, regarding its nature, could not capture patterns as well as textual models.