Adversarial Learning for Infrared and Visible Image Fusion Integrating Semantic Segmentation and Skip Connections
LIU Dongxu
XU Guangyu
Abstract：To address the issues of weakened target information and insufficiently enriched details in infrared-visible image fusion,and to enhance the semantic information retention capability of fused images,a semantic segmentation-skip connections for infrared-visible image fusion(SS-SC-IVIF)adversarial learning model was proposed.At the generator end,the mask with semantic information was obtained through semantic segmentation,and the image was divided into the infrared image target area and the visible light image background area;the two source images and the result of them were used to extract more source image feature information;and the semantic information mask was designed to guide the feature extraction and reconstruction.The two discriminators prompted the fusion results to retain the intensity information of the infrared image and the texture information of the visible image.The results showed that in six objective evaluation indicators except standard deviation(SD),the other five indicators in Netherlands organization for applied scientific research(TNO)data sets and road data sets were improved,of which the entropy(EN)indicator of SS-SC-IVIF model improved by 3.7%and the structural similarity(SSIM)indicator improved by 4.3%compared with the average of the comparison models.The detail information of the source images was maximally retained,and the fusion accuracy was significantly improved by the model.
Keywords：image fusionsemantic segmentationgenerative adversarial networkmask designskip connectionsmultiscale attention network
Publication Date：2025-09-20
Online Publishing Date：2025-09-24(First online date of this platform, not the publication date of the document)
Pages：9( 396-404 )