Few-shot action recognition in video method based on second-order spatiotemporal adaptation
Zhang Bingbing
Li Haibo
Ma Yuanchen
Zhang Jianxin
Abstract：Objectives In the field of few-shot video action recognition,existing methods generally face chal-lenges in adequately processing global spatiotemporal information.These methods typically rely on large amounts of annotated data to train deep models,but with only a limited number of training samples avail-able,they often struggle to effectively capture and utilize the spatiotemporal dynamics in video data.Meth-ods To address this issue,an innovative second-order spatiotemporal adaptive network architecture includ-ing a spatiotemporal adaptive module and a covariance aggregation module was proposed to significantly en-hance the accuracy and robustness of few-shot learning in video action recognition tasks.The spatiotemporal adaptive module dynamically aggregated local and global spatiotemporal information based on changes in video content,thereby optimizing the process of global information extraction.The covariance aggregation module utilized second-order statistical methods to enhance the global spatiotemporal feature representation of videos,providing a more robust global depiction of video content.Results Extensive experiments were conducted on four mainstream video action recognition benchmark datasets.The results demonstrated that the proposed method achieved accuracies of 52.2%and 72.4%for 1-shot and 5-shot tasks on the Something-Something V2 dataset,significantly outperforming the baseline model.Strong performance was also ob-served on Kinetics100,UCF101,and HMDB51 datasets,fully validating its effectiveness and practicality in few-shot video action recognition.Conclusions The proposed second-order spatiotemporal adaptive net-work effectively improved the accuracy and robustness of few-shot video action recognition.It demonstrated significant advantages in processing complex spatiotemporal information.This work provided an innovative and efficient solution addressing critical challenges in spatiotemporal modeling under limited data scenarios.
Keywords：few-shot learningaction recognition in videospatiotemporal representation learningtemporal modelingcovariance aggregation
Publication Date：2025-09-30
Online Publishing Date：2025-08-15(First online date of this platform, not the publication date of the document)
Pages：9( 43-51 )