Segmentação semântica fracamente supervisionada para imagens de fast-food usando arquitetura SegFormer com duplo estudante

Abstract

This study focuses on Semantic Segmentation (SS) of fast-food sandwich assembly images using Deep Learning and weakly supervised learning techniques. Semantic segmentation, which involves pixel-wise classification of the image, is a fundamental task in computer vision and relevant for quality control in franchise assembly. When dealing with food, challenges arise such as the high visual diversity of ingredients and class imbalance; moreover, existing models often rely on public datasets that do not reflect scenarios such as Closed-Circuit Television (CCTV) imagery, which is the focus of this work. Automating quality control by distinguishing ingredients at the pixel level contributes to standardization and consistency. Manual annotation for supervised training is costly and time-consuming, which justifies exploring weakly supervised models that require only image-level labels. We propose a weakly supervised SS model based on Class Activation Maps (CAMs) to identify ingredients in low-resolution images captured by CCTV. The architecture is lightweight and low-resource: it uses SegFormer-B0 in a dual-student setup with mutual supervision via cross pseudo-labels, and incorporates Test-Time Augmentation (TTA) and Dynamic Threshold Adjustment (DTA) for pseudo-label refinement. Results show mIoU of 43.9% (comparable to DuPL, 41.2%), with greater stability across folds (standard deviation 0.6% versus 6.8% for DuPL). The model also exhibits superior computational efficiency, using 33% less VRAM and achieving 1.5–5× higher throughput than DuPL, making it suitable for real-time deployment in resource-constrained environments.