Enhanced gastrointestinal disease classification using a convvit hybrid model on endoscopic images

Endoscopy is a procedure that allows examination of the gastrointestinal system, including the stomach, esophagus, large intestine, and duodenum, with the help of an endoscope. Processing of endoscopic images is important for early detection and treatment of gastrointestinal diseases. In this study, hybrid ConvViT was developed using CNN and ViT to increase the classification accuracy of pathologies in gastrointestinal endoscopic images. CNNs are well-suited for capturing local spatial features through hierarchical convolutions, making them highly effective in detecting fine-grained textures and edge patterns. These capabilities complement the ViT’s global attention mechanism, which excels at modeling long-range dependencies in images. The motivation of this study is to increase the classification accuracy and reliability with the ConvViT model, which was developed by combining the practical features of CNN and ViT models, which are individually successful in different aspects of image processing. The ConvViT model was compared with VGG-16, ResNet-50, Inception-V3 and ViT. Comparable models were tested using a gastrointestinal endoscopic image dataset containing ulcers, polyps, inflammation, bleeding, and regular anatomical features. Experiments showed that ConvViT had better prediction performance than compared models, with 95.87% classification accuracy.