融合主题模型及双语词向量的汉缅双语可比文档获取方法  

Chinese-Burmese Comparable Document Acquisition Based on Topic Model and Bilingual Word Embedding

作　　者：李训宇 毛存礼[1,2] 余正涛[1,2] 高盛祥[1,2] 王振晗 张亚飞 LI Xunyu;MAO Cunli;YU Zhengtao;GAO Shengxiang;WANG Zhenhan;ZHANG Yafei(Faculty of Information Engineering and Automation,Kunming University of Science and Technology,Kunming,Yunnan 650500,China;Yunnan Key Laboratory of Artificial Intelligence,Kunming University of Science and Technology,Kunming,Yunnan 650500,China)

机构地区：[1]昆明理工大学信息工程与自动化学院,云南昆明650500 [2]昆明理工大学云南省人工智能重点实验室,云南昆明650500

出　　处：《中文信息学报》2021年第1期88-95,共8页Journal of Chinese Information Processing

基　　金：国家自然科学基金(61732005,61662041,61761026,61866019,61972186);国家重点研发计划(2019QY1802,2019QY1801);云南省应用基础研究计划重点项目(2019FA023);云南省中青年学术和技术带头人后备人才项目(2019HB006)。

摘　　要：缅甸语属于资源稀缺型语言,汉缅双语可比文档是获取平行句对的重要数据资源。该文提出了一种融合主题模型及双语词向量的汉缅双语可比文档获取方法,将跨语言文档相似度计算转化为跨语言主题相似度计算问题。首先,使用单语LDA主题模型分别抽取汉语、缅甸语的主题,得到对应的主题分布表示;其次,将抽取到的汉缅主题词进行表征得到单语的主题词向量,利用汉缅双语词典将汉语、缅甸语单语主题词向量映射到共享的语义空间,得到汉缅双语主题词向量,最后通过计算汉语、缅甸语主题相似度获取汉缅双语可比文档。实验结果表明,该文提出的方法得到的F1值比基于双语词向量方法提升了5.6%。To collect Chinese-Burmese comparable documents, this paper proposes a acquisition method based on topic model and bilingual word embedding, treating the cross-language document similarity issue as cross-language topic similarity measurement. First, we use the monolingual LDA topic model to extract the Chinese and Burmese topics, respectively, and get the corresponding topics distribution representation. Then, we calculate the topic words for Chinese and Burmese documents, and get the Chinese-Burmese bilingual topic word embedding by mapping the monolingual word embedding into a shared semantic space according the Chinese-Burmese bilingual dictionary. The similarity of Chinese and Burmese document is finally decided by both topic embedding and bilingual word embedding. The experimental results show that the F1 obtained by the proposed method is increased by 5.6% than the baseline using just the word embedding.

关 键 词：主题模型 双语词向量 文档相似度 汉语—缅甸语 双语可比文档 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]