多源文本数据真值发现方法  

Truth discovery method for multi-source text data

作　　者：曹建军 常宸 陶嘉庆 翁年凤 蒋国权 CAO Jianjun;CHANG Chen;TAO Jiaqing;WENG Nianfeng;JIANG Guoquan(The Sixty-third Research Institute,National University of Defense Technology,Nanjing 210007,China;Command and Control Engineering College,Army Engineering University,Nanjing 210007,China;Department of Industrial Engineering,Nanjing Tech University,Nanjing 211800,China)

机构地区：[1]国防科技大学第六十三研究所,江苏南京210007 [2]陆军工程大学指挥控制工程学院,江苏南京210007 [3]南京工业大学工业工程系,江苏南京211800

出　　处：《国防科技大学学报》2022年第4期172-179,共8页Journal of National University of Defense Technology

基　　金：国家自然科学基金资助项目(61371196);中国博士后科学基金资助项目(20090461425);中国博士后科学基金特别资助项目(201003797)。

摘　　要：针对传统真值发现算法无法直接应用于文本数据的问题,提出基于深度神经网络面向多源文本数据的真值发现算法(NN_Truth)。根据文本答案多因素性、词语使用多样性以及文本数据稀疏性等特点,将“数据源-答案”向量作为网络输入,识别答案真值向量作为网络输出,依据真值发现的一般假设,无监督学习各数据源答案向量间关联关系,并最终获得答案真值。实验结果表明,该算法适用于文本数据真值发现场景,较基于检索的方法及传统真值发现算法效果更优。In order to solve the problem that the traditional truth discovery algorithm cannot be applied to text data directly,a truth discovery algorithm(NN_Truth)for text data based on deep neural network was proposed.For the features of multifactorial property of text answers,the diversity of word usages,and the sparsity of the text data,the“source-answer”vector was used as the network input,and the truth vector was recognized as the network output.The relationship between answers from each source could be unsupervised learned according to general hypothesis of truth discovery,and finally obtained the truth.The experiment results show that the proposed algorithm is suitable for text data truth discovery,and it is better than the retrieval methods and traditional truth discovery algorithm.

关 键 词：数据质量 真值发现 神经网络 文本挖掘 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]