结合LDA与Word2vec的文本语义增强方法  

Text Semantic Enhancement Method Combining LDA and Word2vec

作　　者：唐焕玲[1,2,3] 卫红敏 王育林 朱辉 窦全胜 TANG Huanling;WEI Hongmin;WANG Yulin;ZHU Hui;DOU Quansheng(School of Computer Science and Technology,Shandong Technology and Business University,Yantai,Shandong 264005,China;Co-innovation Center of Shandong Colleges and Universities:Future Intelligent Computing,Yantai,Shandong 264005,China;Key Laboratory of Intelligent Information Processing in Universities of Shandong(Shandong Technology and Business University),Yantai,Shandong 264005,China;School of Information and Electronic Engineering,Shandong Technology and Business University,Yantai,Shandong 264005,China;Shanghai Conversation Intelligence Co.Ltd.,Shanghai 200120,China)

机构地区：[1]山东工商学院计算机科学与技术学院,山东烟台264005 [2]山东省高等学校协同创新中心:未来智能计算,山东烟台264005 [3]山东省高校智能信息处理重点实验室(山东工商学院),山东烟台264005 [4]山东工商学院信息与电子工程学院,山东烟台264005 [5]上海绘话智能科技有限公司,上海200120

出　　处：《计算机工程与应用》2022年第13期135-145,共11页Computer Engineering and Applications

基　　金：国家自然科学基金(61976124,61976125,62176140,61873177,61972235,82001775)。

摘　　要：文本的语义表示是自然语言处理和机器学习领域的研究难点,针对目前文本表示中的语义缺失问题,基于LDA主题模型和Word2vec模型,提出一种新的文本语义增强方法Sem2vec(semantic to vector)模型。该模型利用LDA主题模型获得单词的主题分布,计算单词与其上下文词的主题相似度,作为主题语义信息融入到词向量中,代替one-hot向量输入至Sem2vec模型,在最大化对数似然目标函数约束下,训练Sem2vec模型的最优参数,最终输出增强的语义词向量表示,并进一步得到文本的语义增强表示。在不同数据集上的实验结果表明,相比其他经典模型,Sem2vec模型的语义词向量之间的语义相似度计算更为准确。另外,根据Sem2vec模型得到的文本语义向量,在多种文本分类算法上的分类结果,较其他经典模型可以提升0.58%~3.5%,同时也提升了时间性能。Text semantic representation is one of the most difficulty problems in natural language processing and machine learning.To solve the problem of semantic loss in text representation,this paper proposes a new text semantic representation method named Sem2vec(semantic to vector)model which is based on the LDA topic model and the Word2vec model.The topic similarity is calculated according to the word topic distribution obtained by the LDA model.Then the topic semantic word vectors are inputted into the Sem2vec model instead of the one-hot vector.Constrained by maximizing log-likelihood objective function,the parameters of the Sem2vec model are optimized.Finally,the semantic word vectors are learned by the Sem2vec model and the semantic representation of the text is further obtained.The experimental results on different datasets show that compared with the other classic models,the Sem2vec model is more accurate in calculating semantic similarity between words.Moreover,in different classification algorithms,the text semantic vectors generated by the Sem2vec model can improve the text classification results by 0.58%~3.5%and promote the time performance compared with the other classic models.

关 键 词：LDA主题模型 Word2vec模型 语义词向量 语义相似度 文本分类 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]