基于异质图神经网络预训练的多标签文档分类研究  

Pre-training of Heterogeneous Graph Neural Networks for Multi-label Document Classification

作　　者：吴家伟 方全[2] 胡骏 钱胜胜 WU Jiawei;FANG Quan;HU Jun;QIAN Shengsheng(Henan Institute of Advanced Technology,Zhengzhou University,Zhengzhou 450002,China;National Laboratory of Pattern Recognition,Institute of Automation,Chinese Academy of Sciences,Beijing 100190,China)

机构地区：[1]郑州大学河南先进技术研究院,郑州450002 [2]中科院自动化所模式识别国家重点实验室,北京100190

出　　处：《计算机科学》2024年第1期143-149,共7页Computer Science

基　　金：国家自然科学基金(62072456,62036012,62106262)。

摘　　要：多标签文档分类是一种将文档实例与相关标签相关联的技术,近年来受到越来越多研究者的关注。现有的多标签文档分类方法尝试探索文本之外的信息的融合,如文档元数据或标签结构。然而,这些方法要么简单地利用元数据的语义信息,要么没有考虑标签的长尾分布,因此忽略了文档及其元数据之间的高阶关系和标签的分布规律等信息,从而影响到多标签文档分类的准确性。因此,文中提出一种新的基于异质图神经网络预训练的多标签文档分类方法。该方法通过构造文档与其元数据的异质图,采用两种对比学习预训练方法捕获文档与其元数据之间的关系,并通过平衡标签长尾分布的损失函数来提高多标签文档分类的准确性。在基准数据集上的实验结果表明,所提方法的准确率比Transformer提高了8%,比BertXML提高了4.75%,比MATCH提高了1.3%。Multi-label document classification aims to associate document instances with relevant labels,which has received increasing research attention in recent years.Existing multi-label document classification methods attempt to explore the fusion of information beyond the text,such as document metadata or label structure.However,these methods either simply use the semantic information of metadata or do not consider the long-tail distribution of labels,thereby ignoring higher-order relationships between documents and their metadata and the distribution pattern of labels,which affects the accuracy of multi-label document classification.Therefore,this paper proposes a new multi-label document classification method based on the pre-training of hete-rogeneous graph neural networks.The method constructs a heterogeneous graph based on documents and their metadata,adopts two contrastive pre-training methods to capture the relationship between documents and their metadata,and improves the accuracy of multi-label document classification by balancing the problem of long-tail distribution of labels through a loss function.Experimental results on the benchmark dataset show that the proposed method outperforms Transformer BertXML and MATCH by 8%,4.75%,1.3%,respectively.

关 键 词：多标签文档分类 元数据 异质图神经网络 预训练 长尾分布 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]