面向词权重的主题识别应用研究  

Research of Topic Recognition Based on Term Weighting

作　　者：阮光册 夏磊[3] 

机构地区：[1]华东师范大学经济与管理学部信息管理系,上海200241 [2]华东师范大学一带一路与全球发展研究院,上海200241 [3]上海图书馆会展中心,上海200031

出　　处：《情报理论与实践》2019年第12期144-149,共6页Information Studies:Theory & Application

基　　金：上海哲学社会科学一般项目“基于主题模型的学科交叉知识发现研究”的成果之一,项目编号:2016BTQ002

摘　　要：[目的/意义]在文本主题求解时,LDA模型更倾向于高频率的词项,造成主题的语义特征和内容区分度不高.[方法/过程]从文本的词权重入手,综合考虑词项在文本集合中的全局统计特征和局部语义特征,衡量词语在文本中的重要性,并将词语的特征值作为LDA主题模型的输入,改变LDA模型生成词的概率.[结果/结论]实验表明,结合词权重的LDA模型,具有更好的模型拟合度,同时能够较好的识别语料库中主要话题,提高了主题词分布的广度和主题的语义区分度.通过新闻文本数据验证了该方法的可行性与有效性.[局限]对词语的局部语义特征描述需要大数据量的计算.[Purpose/significance]In order to recognize the text topics,LDA model tends to use high-frequency terms,which results in low semantics and content discrimination.[Method/process]Starting from the term weighting,the paper measures importance of words in text,considering both the overall statistical features and local semantic features of words in text set.Then the word feature values are used as input of LDA topic model to change the probability of LDA model generating words.[Result/conclusion]Experiments show that the LDA model combined with term weighting has better model fitting degree,and can identify the main topics in the corpus,and improve the breadth of the distribution of topic words and the semantics of topic.The feasibility and validity of this method are verified by news text data.[Limitations]A large amount of data is needed to describe the local semantic features of words.

关 键 词：主题模型 LDA模型 FTIDF N-GRAM 权重 主题识别 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]