基于高维聚类的文本大数据挖掘算法仿真  

Simulation of Text Big Data Mining Algorithm Based on High Dimensional Clustering

作　　者：郭红建 陈一飞 梅轶群 GUO Hong-jian;CHEN Yi-fei;MEI Yi-qun(School of Information Engineering,Nanjing Audit University,Nanjing Jiangsu 211815,China)

机构地区：[1]南京审计大学信息工程学院,江苏南京211815

出　　处：《计算机仿真》2023年第6期499-503,共5页Computer Simulation

基　　金：国家自然科学基金项目面上项目(72074117);江苏省高校自然科学研究项目面上项目(20KJB630012);江苏省高校人文社会科学研究项目一般项目(2021SJA0351);全国高等院校计算机基础教育研究会2022年立项课题(2022-AFCEC-419)。

摘　　要：文本数据具有规模大,特征维数高等特点。通常含有大量的冗余、空间维度复杂的数据,导致文本大数据信息挖掘困难。因此,提出一种基于高维聚类算法的文本大数据挖掘方法。采用等距离特征映射算法,将多维数据映射到低维空间。通过相空间重建,提取大数据的关键特征。以平均信息熵作为衡量聚类项目的标准,多次不断更新本文聚类中心,当平均信息熵为小数值时,利用密度函数确定原始本文聚类中心,实现文本大数据挖掘。实验结果证明,所提方法的F1值在95%以上,说明文本大数据的聚类精准度高,不会出现过度挖掘问题。Text data is characterized by large scale and high feature dimension and usually contains massive re-dundant data with complex spatial dimensions.Based on high-dimensional clustering algorithm,a method of mining the big data of text was proposed.Firstly,the isometric feature mapping was adopted to map the multidimensional data to low dimensional space.Then,key features of big data were extracted through phase space reconstruction.Secondly,the average information entropy was used as the standard to measure the clustering items.Meanwhile,the clustering center of text was constantly updated many times.When the average information entropy was a small value,the density function was used to determine the original clustering center,so as to realize the text big data mining.Experimental results show that the F1 value of the proposed method is more than 95%,indicating that the clustering accuracy of big data is high and excessive mining will not occur.

关 键 词：聚类算法 平均信息熵 降维处理 相空间重建 文本聚类 特征选择 

分 类 号：TP118[自动化与计算机技术—控制理论与控制工程]