一种基于LDA的潜在语义区划分及Web文档聚类算法  

An Approach of Latent Semantic Space Partition and Web Document Clustering

作　　者：刘振鹿[1] 王大玲[1,2] 冯时[1] 张一飞[1,2] 方东昊[1] 

机构地区：[1]东北大学信息科学与工程学院,辽宁沈阳110819 [2]医学影像计算教育部重点实验室(东北大学),辽宁沈阳110819

出　　处：《中文信息学报》2011年第1期60-65,70,共7页Journal of Chinese Information Processing

基　　金：国家自然科学基金资助项目(60973019);国家863计划资助项目(2009AA01Z131)

摘　　要：该文应用LDA模型进行文档的潜在语义分析,将语义分布划分成低频、中频、高频语义区,以低频语义区的语义进行Web游离文档检测,以中、高频语义区的语义作为文档特征进行文档聚类,采用文档类别与语义互作用机制对聚类结果进行修正。与相关工作比较,该文不仅应用LDA模型表示文档,而且进行了深入的语义分布区域划分,并将分析结果应用于Web文档聚类。实验表明,该文提出的基于LDA的文档类别与语义互作用聚类算法获得了更好的聚类结果。This paper applies the LDA model to analyze latent semantics of documents and partition the semantic space into low,middle and high frequency space.The semantics in low frequency space are used to detect outlier web documents.The semantics in middle and high frequency space are devoted to document clustering as features of the documents.The quality of clustering results is improved by a mutual-action mechanism between document clusters and semantics.Compared with related work,this paper not only applies LDA model to represent documents,but also analyzes the semantic distribution in depth and applies the results of analysis to web document clustering.Experiments show that the clustering algorithm of the mutual-action between LDA-based document class and semantic in this paper deserve better effects in document clustering.

关 键 词：LDA 潜在语义 语义分布 文档聚类 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]