Global Spatial-Temporal Information Encoder-Decoder Based Action Segmentation in Untrimmed Video  

作　　者：Yichao Liu Yiyang Sun Zhide Chen Chen Feng Kexin Zhu 

机构地区：[1]Department of Computer and Cyberspace Security,Fujian Normal University,Fuzhou 350007,China [2]Department of Information Engineering,Sun Yat-sen University,Kaohsiung 80424,China

出　　处：《Tsinghua Science and Technology》2025年第1期290-302,共13页清华大学学报自然科学版(英文版)

基　　金：supported by the National Natural Science Foundation of China(No.62277010);the Fuzhou-Xiamen-Quanzhou National Independent Innovation Demonstration Zone Collaborative Innovation Platform Project(No.2022FX6);the Fujian Provincial Health Commission Technology Plan Project(No.2021CXA001).

摘　　要：Action segmentation has made significant progress,but segmenting and recognizing actions from untrimmed long videos remains a challenging problem.Most state-of-the-art methods focus on designing models based on temporal convolution.However,the limitations of modeling long-term temporal dependencies and the inflexibility of temporal convolutions restrict the potential of these models.To address the issue of over-segmentation in existing action segmentation methods,which leads to classification errors and reduced segmentation quality,this paper proposes a global spatial-temporal information encoder-decoder based action segmentation method.The method proposed in this paper uses the global temporal information captured by refinement layer to assist the Encoder-Decoder(ED)structure in judging the action segmentation point more accurately and,at the same time,suppress the excessive segmentation phenomenon caused by the ED structure.The method proposed in this paper achieves 93%frame accuracy on the constructed real Tai Chi action dataset.The experimental results prove that this method can accurately and efficiently complete the long video action segmentation task.

关 键 词：Encoder-Decoder(ED) Bidirectional Long Short-Term Memory(BiLSTM) Tai Chi action segmentation untrimmed video 

分 类 号：O17[理学—数学]