基于事件异构图表示的文本去重算法  

Text Deduplication Algorithm Based on Event Heterogeneous Graph Representation

作　　者：艾玮 许佳 谢灿豪 孟涛 AI Wei;XU Jia;XIE Canhao;MENG Tao(School of Computer and Information Engineering,Central South University of Forestry and Technology,Changsha 410018,China)

机构地区：[1]中南林业科技大学计算机与信息工程学院,湖南长沙410018

出　　处：《湖南大学学报（自然科学版）》2023年第2期74-85,共12页Journal of Hunan University:Natural Sciences

基　　金：国家自然科学基金资助项目(61802444)。

摘　　要：基于图结构的文本表示方法在新闻文本去重中具有更好的效果.但是,目前该表示方法还不能完整地表示文本的全部信息,并且忽略了图的语义信息,降低了新闻文本的去重效果.为此,本研究提出基于事件异构图表示的文本去重算法,该算法首先通过事件异构图表示新闻文本的全局语义与结构信息,然后提出双标签图核算法表征事件异构图,实现深度表征图的结构及语义信息.实验结果表明,该研究提出的去重算法比现有的基于图结构的文本表示去重方法在F1-score指标上提升了10%.最后,该算法能提高新闻文本的去重效果.The text representation method based on graph structure has a better effect in news text deduplication. However, at present, this representation method cannot fully represent the complete information of the text, and ignores the semantic information of the graph, which reduces the deduplication effect of news text. To this end, this study proposes a text deduplication algorithm based on event heterogeneous graph representation. The algorithm first represents the global semantic and structural information of news text through event heterogeneous graph, and then proposes a dual-label graph kernel algorithm to represent event heterogeneous graph to realize the structure and semantic information of the deep representation graph. The experimental results show that the deduplication algorithm proposed improves the F1-score index by 10%, compared with the existing text representation deduplication method based on graph structure. Finally, the algorithm can improve the deduplication effect of news text.

关 键 词：图核算法 事件异构图 新闻文本 文本表示 文本去重 

分 类 号：T391.1[一般工业技术]