基于多模态相似融合的新闻视频故事分割算法  

News video story segmentation algorithm based on multi-mode similarity fusion

作　　者：吴培 周激流[1] WU Pei;ZHOU Jiiu(College of Electronic Information,Sichuan University,Chengdu 610065,China)

机构地区：[1]四川大学电子信息学院,成都610065

出　　处：《智能计算机与应用》2024年第1期70-75,84,共7页Intelligent Computer and Applications

摘　　要：新闻视频数量的不断增加,为准确分割用户感兴趣的新闻视频,本文提出了一种基于多模态相似融合的新闻视频故事分割算法。首先,通过选定视频切割点获取候选新闻故事单元边界,将视频分成音频流和视频流;其次,选择静音区间为音频候选切分点,主持人镜头帧和主题字幕帧作为视频候选切分点,根据候选切分点获得新闻故事基本单元,利用语义相似性分析各单元内容进行合并或独立分离,得到最终新闻故事;最后,采用人脸识别、YOLOv5来进行主题字幕检测、语义相似性合并或独立新闻故事基本单元,使得新闻故事边界划分更为准确。该新闻视频故事分割算法在《新闻联播》视频中查全率和查准率分别达到了97.17%和98.19%,为新闻视频导航、检索等应用提供辅助准备。With the increasing number of news videos,in order to accurately segment news videos of interest to users,this paper proposes a news video story segmentation algorithm based on multimodal similarity fusion.Firstly,by selecting video cutting points to obtain candidate news story unit boundaries,the video is divided into audio and video streams;Secondly,select the silent interval as the audio candidate segmentation point,and the host lens frame and theme subtitle frame as the video candidate segmentation points.Based on the candidate segmentation points,obtain the basic units of the news story,and use semantic similarity analysis to merge or separate the content of each unit separately to obtain the final news story;Finally,facial recognition and YOLOv5 are used for topic subtitle detection,semantic similarity merging,or independent news story basic units to make news story boundary division more accurate.The recall and precision of the news video story segmentation algorithm in CCTV News video reached 97.17%and 98.19%respectively,providing auxiliary preparation for news video navigation,retrieval and other applications.

关 键 词：新闻故事基本单元 主题字幕 人脸识别 YOLOv5 语义相似性 

分 类 号：TP399[自动化与计算机技术—计算机应用技术]