Long Text Classification Model Based on XLNet and Deep Convolution Aggregation
QU Chenghao
XUE Tao
HU Weihua
Abstract：In the long text classification task,the pre-trained language model is limited by the input length and cannot obtain the long distance semantic information,while truncating the long text will lose the association information between segments.To solve this problem,this paper proposes an aggregation classification model for truncated text.Firstly,the semantic vector of truncat-ed text is obtained by XLNet language model,and the sequential information of segments is extracted by BiLSTM,and the semantic vector of segments is aggregated by multi-channel deep convolution(MDCNN).MDCNN uses hierarchical pooling to compress the sequence length of segmented text,and uses multi-channel convolution to obtain the correlation features of multi-segment text.In order to solve the problem of large model parameters caused by deep network,MDCNN uses isometric convolution to extract the se-mantic features of long text,fixed feature dimensions to reduce the loss of semantic information,and uses linear pre-activation func-tion to carry out residual connection,avoiding the process of dimension mapping in residual connection and making the model reach deeper network layers.The experiment compares several commonly used text classification models,the model can effectively model long text data,and has good classification effect.
Keywords：text classificationXLNetBiLSTMCNNresidual connection
Publication Date：2024-09-20
Online Publishing Date：2025-08-15(First online date of this platform, not the publication date of the document)
Pages：6( 2640-2644,2674 )