Human Action Recognition Based on Time Series Allocation and Multi-frequency Component Compression

SHE Benjie; ZHU Yanmin; SONG Jian; HUA Jian

doi:10.13501/j.cnki.42-1908/n.2024.06.025

Abstract：In order to address the issues of time-series feature extraction and feature compression in three dimensional convolutional neural network(3D-CNN)-based human action recognition,an improved network stronger 3D-CNN-based approach for skeleton-based action recognition(S-PoseConv3D)was proposed.The network effectively improved the performance of the original PoseConv3D by introducing the time series allocation strategy(TSAS)and the multi-frequency component feature compression fusion(M3F).TSAS dynamically adjusted the weights of different time series in the feature extraction process,and calculating the weights of the time series to enhance the model′s ability to capture the spatial-temporal features of human behaviors.Meanwhile,the M3F utilized the discrete cosine transform to compress and fuse the multi-frequency components in the feature map,thereby retaining the spatial and temporal features of human behaviors and reducing feature loss due to global pooling.The network was experimented on the dataset built on top of gymnastic videos(FineGYM)and Nanyang university of technology skeleton behavior recognition dataset 60 classification extended subset(NTU60-XSub),and the results showed that the improved network improved the mean top-1 accuracy by 3.93%on the FineGYM dataset compared to the original PoseConv3D.It also demonstrated significant performance improvement on the NTU60-XSub dataset,which indicated higher accuracy and robustness,and it was well-suited for application in the field of human behavior recognition.This network can be applied in such fields as security monitoring and human-computer interaction.