基于R(2+1)D三元孪生网络的短视频指纹提取  

Short Video Fingerprinting Extraction Based on R(2+1)D Triplet Siamese Networks

作　　者：王逸舟 张菁 张淑莹 卓力 WANG Yi-zhou;ZHANG Jing;ZHANG Shu-ying;ZHUO Li(Faculty of Information,Beijing University of Technology,Beijing 100124,China;Beijing Key Laboratory of Computational Intelligence and Intelligent System,Beijing 100124,China)

机构地区：[1]北京工业大学信息学部,北京100124 [2]计算智能与智能系统北京市重点实验室,北京100124

出　　处：《测控技术》2022年第4期11-18,27,共9页Measurement & Control Technology

基　　金：国家自然科学基金(61971016);北京市自然科学基金-市教委联合资助项目(KZ201910005007)。

摘　　要：随着自媒体时代的兴起,网民自制的短视频数据在网络上广泛传播,视频版权保护已成为重中之重。视频指纹技术将数字视频内容映射为唯一的身份描述符,用于视频数据的智能化审核。如何充分挖掘短视频的时空信息和视频间的关联性形成有效的视觉特征表达,是决定短视频指纹质量的关键因素。因此,基于R(2+1)D三元孪生网络模型,提出一种短视频指纹提取方法。首先,使用R(2+1)D卷积神经网络模型提取短视频的时空特征;然后构建权重参数共享的三元组网络学习成组视频的关联性,映射为紧凑的哈希特征表示;最后通过哈希层编码为视频指纹。在CC_Web_Video和VCDB数据集进行了实验,结果表明该方法可以在保证短视频指纹紧凑性的前提下,取得优于其他算法的性能指标。With the rise of “We Media” era, short video data created by Internet users are widely spread on the network, and video copyright protection has become a top priority.Video fingerprinting technology represents digital video content as a unique identity descriptor for intelligent audit of video data.How to fully explore the spatiotemporal information of short videos and the correlation between videos to form an effective visual feature representation is the key factor to determine the quality of short video fingerprinting.Therefore, a short video fingerprinting extraction method based on R(2+1)D triplet siamese network model is proposed.Firstly, the R(2+1)D convolution neural network model is used to extract the spatiotemporal feature of short video.Then, a triplet network with shared weight parameters is constructed to learn the correlation of group video, which is mapped into a compact hash feature representation.Finally, the video fingerprintings are encoded through the hash layer.Experimental results on CC_Web_Video dataset and VCDB dataset show that the proposed method can achieve competitive performance than other methods under ensuring the compactness of video fingerprinting.

关 键 词：短视频指纹提取 R(2+1)D 孪生网络 三元组损失 时空特征 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]