Integrating ResNet-50 and Vision Transformer Architectures for Robust and Efficient Tomato Fruit Ripeness Classification

Abstract

This article introduces a new hybrid deep learning framework that combines ResNet-50 and Vision Transformer (ViT) models to classify tomato fruits by ripeness and quality. The hybrid model takes advantage of the ResNet-50's capability to extract features in local spatial regions. Additionally, it combines ResNet-50 with ViT's ability to establish a global contextual relationship between elements with self-attention. The framework overcomes the limitations of utilizing a single model. The framework was trained and evaluated, using a balanced and diverse dataset. It consists of four tomato classes, ripe, unripe, unevenly ripened, and damaged, collected under controlled conditions in the field. The experiments conducted aggressively in the study demonstrated a high classification accuracy of 98% with a hybrid framework compared to individual ResNet-50 and ViT models. The results in addition to the general performance were validated further using precision, recall, the area under the curve (AUC) and the F1-score. The study also included a computational efficiency test to look for accuracy versus multiple time and spatial resource costs. The research successfully addresses the challenges with dataset diversity, computational costs, and real-time feature deployments through the conceptualization of strategies (data augmentation, transfer learning) to improve generalization. The hybrid framework is expected to provide similar design principles and measures for use with other agricultural products.  It can help in real-time sorting applications in variable real-world scenarios in agricultural operations. The research provides a pathway for developing smarter, scalable, and sustainable solutions in food quality and safety systems within precision agriculture. Future work will incorporate deployment and optimize the framework for edge devices in a real-world variable farm environment.