Multi-task Cross-domain Migration Semantic Segmentation Combined with Self-supervised Depth Estimation

LI Zhaoyi; SHI Chaoxia; WANG Yanqing

doi:10.3969/j.issn.1672-9722.2025.10.006

Abstract：Supervised semantic segmentation usually requires a large number of sample annotations,and the use of migration learning can improve the generalization ability of segmentation models under different domains.Most domain migration semantic seg-mentation methods only utilize the semantic labels of the source domain and ignore other available information.Considering that im-age semantics and depth information may be intrinsically related,a domain migration semantic segmentation method combining self-supervised depth estimation is proposed.The method consists of two modules,a prediction network module and an inter-do-main alignment network module.The semantic segmentation branch of the prediction network is based on a multiscale fusion net-work model,and the depth estimation and bit pose estimation branches are effectively supervised by inter-frame consistency con-struction,while at the same time,each prediction sub-network uses a common encoding network for extracting features common to multiple tasks.Inter-domain alignment is achieved using generative adversarial networks,and inter-domain migration is achieved by blurring the differences between the prediction results of the original and target domains,and the entropy mapping of semantic and depth information is explicitly constructed as the input of the adversarial discriminator on this basis to further improve multi-task cross validity.Key words semantic segmentation,depth estimation,convolutional neural network,transfer learning.The final results are tested on Cityscapes,KITTI,SYNTHIA and other datasets,and 62.8%of mIoU is achieved with SYNTHIA as the original domain and Cityscapes as the target domain.The effectiveness of the method is demonstrated by the 52.2%mIoU achieved with CARLA as the original domain and KITTI as the target domain in the dataset.