Bridging the Gap: Evaluating the Generalizability of Simulated Data for Urban Feature Classification Using Deep Learning

In this research, we present a comprehensive analysis of urban feature classification using Deep Learning (DL), with a focus on the performance of simulated data. We construct a detailed 3D model of the IISER Bhopal campus, from which synthetic data is generated, and compare it against real-world data captured using a 360-degree view camera for capturing the hierarchical information in the images used for image segmentation. The OCRNet segmentation architecture is applied to both datasets, with hyperparameter tuning to optimize performance. Our experiments assess the efficacy of simulated data in feature extraction and the model's generalizability when trained on synthetic data and tested on real-world scenarios. We discuss the model's performance, highlighting the classes where it fails to generalize, and analyze the potential causes, such as lighting, textures, and contrast. We also explore the impact of data augmentation strategies to reduce bias and domain gaps, evaluating the extent to which these adjustments facilitate adaptation to real data. We achieved 85.39% MIoU on 30% augmented data. Our findings contribute to understanding the distribution gap's impact on model adaptation and generalization across complex outdoor datasets. This study not only advances the field of computer vision but also provides insights into the potential and limitations of using simulated data for DL applications in urban feature classification.