3D multimodal brain image reconstruction algorithm based masked modeling technology
JIA Haojing
LI Xiang
XU Yunfeng
WEI Benzheng
Abstract：To address the issue that spatial information in 3D brain image data is complex and difficult to extract effective features,we proposed a 3D multimodal brain image reconstruction algorithm 3MbiRMA based on masked image modeling technology.A dual-branch structure was adopted to extract features of diffusion tensor imaging(DTI)and magnetic resonance imaging(MRI)3D brain image,to fuse multimodal features through feature decoupling,and to realize image reconstruction based on the decoder.Furthermore,the masking strategy and squeeze-space attention mechanism could not only significantly reduce information redundancy and enhance feature effectiveness,but also effectively reduce the computational complexity.Experimental results on the BeijingEN and ADNI data-sets indicated that the computational complexity of the 3MbiRMA was only 1/4 of that of the classical masked autoencoder(MAE)mod-el.Compared with the classical 3D reconstruction algorithms(3D UNet,VNet),the computational complexity of the 3MbiRMA was al-so significantly reduced.The algorithm can significantly improve the reconstruction performance and provide technical support for the re-search of brain image reconstruction.
Keywords：Image reconstructionMultimodalMask image modelingFeature extraction3D brain imaging
Publication Date：2025-06-30
Online Publishing Date：2026-09-11(First online date of this platform, not the publication date of the document)
Pages：12( 150-161 )