QH-K:面向新闻文本主题抽取的改进H-K聚类算法  

QH-K:improved H-K clustering algorithm for news text topic extraction

作　　者：杨玉娟[1] 冯霞 王永利[2] YANG Yujuan;FENG Xia;WANG Yongli(Nanjing Library,Nanjing 210018,China;School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing 210094,China)

机构地区：[1]南京图书馆,江苏南京210018 [2]南京理工大学计算机科学与工程学院,江苏南京210094

出　　处：《南京邮电大学学报（自然科学版）》2020年第1期82-88,共7页Journal of Nanjing University of Posts and Telecommunications：Natural Science Edition

基　　金：国家自然科学基金(61170035,61272420,81674099);江苏省“六大人才高峰”高层次人才项目(WLW-004);中央高校基本科研业务费专项资金(30916011328,30918015103);南京市科技发展计划(201805036);江苏省科技成果转化专项资金(BA2013047);提升政府治理能力大数据应用技术国家工程实验室2017-2018年度开放基金资助项目。

摘　　要：随着网络信息文本的爆发式增长,人们从繁多的新闻中获取特定有效的信息变得愈发困难。在大数据处理中,学者们经常使用文本聚类方法作为新闻主题提取和趋势跟踪的主要措施。针对凝聚型层次聚类算法和K-Means算法在文本聚类上的优势和缺陷,提出一种新的新闻文本聚类优化处理算法——QH-K(K-Means based on Quick Hierarchical Clustering)算法。首先,通过word2vector模型训练文本得到词向量;其次,采用优化的凝聚型层次聚类算法对文本聚类,并根据优化处理算法所提出聚类有效性指标ST得到初始聚类个数和聚类中心;最后,引入K-Means算法对聚类结果进行优化,提高最终聚类的效果。实验证明,QHK聚类优化处理算法的正确率、召回率、F值相比传统算法都得到了一定程度的提升;此外,算法的运行时间也有所下降。With the explosive growth of online information texts,it becomes more and more difficult for people to get specific and effective information from various news reports.In big data processing,scholars often use text clustering method as the main measure for news topic extraction and trend tracking.For condensing hierarchy clustering algorithm and K-Means algorithm on text clustering advantages and defects,put forward a new news text clustering optimization algorithm—QH-K(K-Means based on the quick hierarchical clustering)algorithm.Firstly,word vector is obtained by training text through word2vector model.Secondly,the optimized agglomeration hierarchical clustering algorithm was used to cluster text,and the clustering validity index ST proposed by the optimization algorithm was used to get the initial number of clustering and the clustering center.Finally,K-Means algorithm is introduced to optimize the clustering results and improve the final clustering effect.The experiment shows that the accuracy,recall rate and F value of QH-K clustering optimization algorithm are all improved to a certain extent compared with the traditional algorithm.In addition,the running time of the algorithm is also reduced.

关 键 词：文本聚类 凝聚型层次聚类 K-MEANS 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]