联合场景和行为特征的短视频行为识别  

Short Video Behavior Recognition Combining Scene and Behavior Features

作　　者：董旭 谭励[1] 周丽娜 宋艳艳 DONG Xu;TAN Li;ZHOU Lina;SONG Yanyan(School of Computer and Information Engineering,Beijing Technology and Business University,Beijing 100048,China)

机构地区：[1]北京工商大学计算机与信息工程学院,北京100048

出　　处：《计算机科学与探索》2020年第10期1754-1761,共8页Journal of Frontiers of Computer Science and Technology

基　　金：北京市自然科学基金No.4172013;北京市自然科学基金-海淀原始创新联合基金No.L182007;国家自然科学基金No.61702020。

摘　　要：目前行为识别方法更关注动作本身,但短视频中包含的信息比较少,需要利用视频中的多种特征信息,提高任务行为识别的准确率。因此,对基于场景和行为联合特征的短视频行为识别方法进行了研究,利用场景信息作为上下文信息,提高传统单一行为识别网络的效果。首先对短视频中的场景特征利用深度融合网络进行提取;然后对短视频中的行为特征利用可变卷积网络进行RGB特征和Flow特征提取;最后利用字典学习的方法对构建的联合特征进行稀疏表示,提取出更具解释性的特征信息。在Charades测试集top-5准确率为33%,优于传统单一行为识别网络,使行为识别效果更加准确。The behavior recognition method pays more attention to the action itself,but the short video contains less information.And it is necessary to utilize various feature information in the video as much as possible to improve the accuracy of behavioral recognition.Therefore,the short video behavior recognition method based on scene and behavior joint features is studied,and the scene information is used as context information to improve the effect of traditional single behavior recognition network.First,the scene features in the short video are extracted using a deep fusion network.Then,the behavioral features in the short video utilize the variable convolutional network for RGB features and flow features extraction.Finally,the dictionary learning method is used to sparsely represent the joint features,and more explanatory feature information is extracted for short video behavior recognition.The top-5 accuracy rate in the Charades test set is 33%.It is superior to the traditional single behavior recognition network,making the behavior recognition effect more accurate.

关 键 词：场景识别 行为识别 字典学习 深度学习 视频理解 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]